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THE UNITED STATES AIR FORCE 
OCCUPATIONAL RESEARCH PROJECT 



I . INTRODUCTION 



The United States Air Force Occupational Research Project was 
established In 1958, some IS years ago, and has been supported con* 
tlnuously since that time. The project objectives call for the 
development of methodologies In a number of areas. Including the 
following: 

Job Analysis (Collection, analyses, and reporting of Information 
defining work performed by personnel) 

Job Evaluation (for grade, pay and skill levels) 

Job Structures (Including job engineering, work organization, 
and occupational qlasslf Icatlon) 

Job Requirements (for aptitude, training experience, etc.) 

Career Development 

Personnel Utilization 

Job Satisfaction (In particular, as It relates to retention) 

Obviously, there's no way I can discuss even the highlights 
of a 15-year program In the limited time we have today. What I 
propose to do Is to make some observations about the techniques 
the Air Force has developed for collecting, analyzing, and reporting 
occupational data; then I will discuss a few recently completed or 
ongoing studies in other occupational research areas which may be 
of Interest to you. Since this is an Informal survey paper, I 
will draw freely from previously published papers, my own memory, 
and data from studies yet to be published. My apologies to those 
of you who are already familiar with our job survey procedures, 
but I feel I must go into some detail describing this technology 
to participants here who have not had access to our in-service 
report series. 
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II. DEVELOPMENT OF JOB SURVEY TECHNIQUES^ 

Reasons for selecting the Job Inventory Approach 

In the Air Force we chose the job inventory as the only feasi- 
ble approach for collecting work-task information from large numbers 
of workers. There were a number of reasons for this decision: 
First , the technique is economical. Data can be collected from 
thousands of people throughout the Service for less than it would 
cost to collect data on .a few people using professional Job analysts. 
Second , the information obtained using job inventories is quanti- 
fiable. That is, you can actually count the number of people per- 
forming any particular task, and describe their characteristics. 
Note that data collected by traditional job analysis are not quanti- 
fiable. No two analysts will describe a job in exactly the same 
terms. Third, the fact that information collected with job inven- 
tories is quantifiable means that it can be stored, manipulated, 
analyzed, and reported by computer. Finally , the fact that infor- 
mation is quantifiable also means that it can be validated and 
checked for stability using conventional statistical techniques. 

Now let me describe a job inventory. It contains two sections. 
The first section has questions to be answered by a worker about 
his job and himself - questions relating to name, identification 
number, previous education, time-on-the-job, tools used, job loca- 
tion, equipment worked on, training schools, pay grade, job atti- 
tudes, and so on. Any item can be included in the background infor-* 
mation section of an inventory which may help answer questions posed 
by managers of the personnel system. The second section of a job 
inventory is simply a list of all the significant tasks that may 
be performed by workers in the occupational area to be surveyed. 
That is, it includes tasks being performed by apprentices, journey- 
men, first-line supervisors, and superintendents in one or more 
occupations, such as supply specialist or engine mechanic. If the 
task list is properly constructed, and this point is important to 
understand, then every worker in the occupation should be able to 
define his job adequately in terms of a subset of tasks in the 
inventory.. 

Construction of Job Inventories 

Let me describe some of our experiences in constructing and 
administering job inventories. Ordinarily, an initial task list 
is constructed from available printed materials. In the Air Force 
program, this list is first reviewed by 5 to 10 senior supervisors 
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in an interview situation; they correct technical wording and add 
additioaal tasks which they know are being performed by worke :s in 
their occupational area. This expanded task list is then sent by 
mail for a field review by supervisors at various locations 
throughout the Air Force* According to the complexity of the 
occupational area, these mail reviews may be obtained from as few 
as 25 to as many as 100 supervisors. At some time during the 
construction phase, the task list is also reviewed by technical 
school Instructors. The final task list is arrived at through 
this iterative process. 

There is some variation in construction techniques used ^by the 
military services. For example, the Marine Corps does not use a 
mail rcviey procedure, but makes extensive use of personal inter- 
views at many locations. The Army makes use of technical school 
instructors as inventory constructors. The Coast Guard, which also 
constructs and administers inventories, essentially follows the 
Air Force techniques. 

Air Force experiences have led to two conclusions. First s 
individuals who are untrained in writing task statements do a poor 
job of building job Inventories for their own occupational area. 
It is better to keep the pencil in the hands of a trained inventory 
constructor and let supervisors in the field of interest serve only 
as technical advisors. Second , if inventories are constructed by 
technical school instructors, care must be taken to see that they 
are not biased through Inclusion of only those tasks which have 
relevance for training. For example, a task concerned with sweep- 
ing the floor has little relevance for training^ but may have a 
great deal of relevance for managers interested in job satisfaction, 
job evaluation, or job reengineering. It is best to have inven- 
tories constructed by individuals who have a broad perspective of 
all future applications of occupational data. 

How many tasks should be included in an inventory? This has 
been a major problem faced by every organization entering the Job 
survey business, t can only report what I believe to be a common 
experience. Most agencies begin with inventories which are too 
short. Ten years ago, the Air Force inventories were averaging 
250 to 350 tasks. Today they are averaging around 500 tasks or 
more. Yet the Air Force has relatively narrow occupational career 
ladders - approximately 230 of them. Inventories constructed by 
smaller military services tend to be much longer. In the 
Australian Air Force, for example, job inveintorles sometimes con- 
tain more than 1,000 tasks. I realize that such lengthy instruments 
may appear to be a problem, but they are not as difficult to manage 
as one might think. If task statements are organized under duty 



headings, and if the worker has to mark only those tasks which he 
actually performs, then even a long inventory can be filled out in 
a reasonable period of time. Furthermore, it has been a common 
finding that detailed task lists lead to firmer conclusions concern- 
ing such things as the establishment of training requirements and 
the evaluation of occupational categories. 

How many background questions are normally included in an 
inventory ? Again, we have found more and more uses for background 
information. It is extremely Important to be able to define any 
subgroup of people which may be of interest to management. If a 
manager wants to know the tasks being performed by aircraft mechanics 
working on a particular aircraft at particular locations who have 
taken certain training and who have been on the job less than one 
year, this can be obtained only if background variables have been 
included which define the relevant characteristics. For reasons to 
be discussed later, the single most important background variable 
for inclusion in a job inventory is worker identification. 

How many workers should be sampled in an occupational area ? The 
more the better. If one were interested only in the occupation as a 
whole, then perhaps a small sample would suffice. But experience 
has shown that managers are often interested in definable groups 
such as females, individuals at a particular grade or salary level, 
workers maintaining a particular tjrpe of equipment, and so on. 
Unless one has collected information from a large sample, then 
there will be insufficient numbers of cases to make reliable infer- 
ences about such groups of interest. Large samples are also needed 
to perform meaningful job-typing analyses — especially if the 
occupational area is complex. 

The £omprehensive Occupational Data jtoalysis jPrograms package 
(which we call CODAP) is designed to handle data on samples of 
20,000 workers, except for programs associated with job-typing 
analyses, which will now accept data on 7,000 workers. In the 
Air Force we have attempted to obtain 100% samples in occupational 
areas containing 2,000 or fewer workers. In larger occupational 
areas, ^. . have attempted to ''btain data on not fewer than 2,000 
workers. If the occupational area is known to contain a variety 
of job types, we may obtain data on 5,000 or more workers. 

What about the costs of data collection and analysis? This 
is a fair question, especially when one considers administering 
long inventories to many workers. The cost of developing an inven- 
tory and of analyzing the results is essentially the same, regard- 
less of the length of the inventory or the number of persons to 
whom it is given. It can cost between one and two hours of work 
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time for each worker included in the survey, which is of conse- 
quence. However, in the Air Force, inventories are administered 
so as not to interfere with accomplishment of primary mission, so 
the costs and value are weighed against the costs and value of other 
nondirect mission programs which consume time, such as commander's 
calls, formations, physical training, and so on. 

One cost, which can be substantial, is that of getting the 
response information onto magnetic tape, ready for computer input. 
All military services are, or soon will be, collecting data on opti<- 
cal scanning sheets. To the extent that data are processed by scanner 
the costs of preparing data for computer input is reasonable. 

I realize that Z have not given you a specific answer about costs 
but I can assure you that costs are modest compared with the savings 
which can be generated from appropriate applications of occupational 
data. I will address this topic directly a little later. 

Administration of Job Inventories 

Now let us turn our attention for a moment to the problem of 
inventory administration. In the Air Force, inventories are sent in 
bulk to Consolidated Base Personnel Offices throughout the world. 
Instructions specify that workers meeting certain specifications 
will be called into testing rooms to fill out inventory forms under 
controlled conditions. In the Marine Corps, the task analysis tinit 
sends out teams to administer inventories on site at various loca-* 
tions. They report excellent results. However, this approach is 
feasible only if a Service or organization has a limited nimiber of 
bases or Installations. 

Instructions for filling out an inventory are relatively simple. 
The worker completes the background section; reads the task list and 
checks those tasks which he performs as part of his normal Job; 
writes in any significant tasks which he performs which were not 
in the task list; and then rates the tasks he has checked using a 
relative tlme~spent scale. 

The write-in feature serves several useful purposes, but pri- 
marily it provides an indication of the quality of the task list. 
If a large number of significant new tasks are uncovered by the 
write- in feature, then the administration of a supplementary survey 
may be required; otherwise the uncovered tasks are used to guide 
interpretation of results and are saved for inclusion In the next 
form of the survey Instrument. 



Quality of Job Inventory Data 

Perhaps the most Important question which needs to be answered 
at this point Is this : Can workers be trusted to be thoroug h and 
comple tely honest when they fill out job Inventories ? Studies have 
been conducted concerning this iquestlon» and I can say that the 
answer Is definitely **Yes,** at least as far as workers In the 
Air Force are concerned. We know that when a worker fills out an 
Inventory on two occasions, he gives essentially the same Informa* 
tlon both times. Split-half reliabilities tor Information such as 
the percent of workers performing various tasks run from .95 to .99. 
Supervisors agree with the Information provided by their subordln*- 
ates. Information collected with dally work records Is consistent 
with Information collected with Inventories. Workers do not In- 
flate their job descriptions In terms of the number and difficulty 
levels of tasks they report. The work tasks reported by Indi- 
viduals are consistent with the Information they provide in the 
background section concerning tools utilized and equipment worked on. 

Many such studies have been conducted and reported. However, 
the experiences which have convinced us beyond any doubt that we 
are getting high-quality information are less objective and have 
never been fully documented. For example, during the first several 
years we obtained the telephone number of every worker who filled 
out an inventory. When we received what we thought might be false 
information, we called the worker and talked with him about his 
job. Over and over again, we found the worker was trying to be 
honest. Most often, the worker had been assigned a peculiar job 
because of local circumstances. In some Instances we found our 
inventory contained bad task statements which did not allow the 
worker to reflect his true job. We did find that, while being 
honest, many workers will give themselves the benefit of the doubt. 
For example, a worker might claim to perform a task when, in fact, 
he only performs part of that task. This is one of the problems 
with task statements which are too broad, and it helps to explain 
why our inventories now have over 500 task statements. 

Another factor which helps us to feel confident about our 
data is that we have published analysis results from over 200,000 
cases in approximately 150 occupational areas, and these results 
have never been proved wrong by managers, workers, or trainers in 
those occupational areas. I will have to admit that there have 
been occasions when we were worried. In one instance, we found that 
very few workers were performing a large set of tasks which consti- 
tuted approximately 25% of a training course. The managers of the 
occupational area were so unbelieving that they did an Independent 
survey In which every worker in the occupational area was inter- 
viewed to see if, in fact, he performed any of the tasks in question. 
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The results of this interview-survey were for all practical purposes 
identical with those obtained from the inventory administration. 
Experiences like this have convinced not only the researchers, but 
also Air Force management, that job inventories yield good data. 

Our latest experience with the power of job inventories to 
give quality data came when we surveyed approximately 5,000 civilian 
workers in one occupational area. We were particularly worried in 
this instance, since civilian pay is directly tied to job content. 
Under this circumstance, a worker might feel he has something to 
gain by being dishonest, or something to lose by being honest. We 
are pleased to report that analyses indicate that, even under this 
condition, workers are. honest. 

We feel that there are two factors operating which cause us 
to get honest reports from workers, and that these factors are 
Interacting. First > we ask the worker to provide his name and 
social security number in the inventory, and second, the informa- 
tion he provides is objectively verifiable. It is unlikely that a 
worker will claim to perform a task when everyone around him knows 
that he does not perform that task. Similarly, it is unlikely 
that he will fail to report a task which everyone around him knows 
he performs. 

Importance of Worker Identification 

There are several reasons why I strongly recommend that name 
and identification information be obtained from workers who fill 
out job inventories. First, we have conducted many studies demon- 
strating that high-quality data can be obtained when workers provide 
their names . If identification information is not obtained, one 
cannot even conduct a study to validate his data. Second , collect- 
ing identification information enables one to follow-up workers and 
trace their career development over time. Third, identification 
information can be used to match with other personnel files to pick 
up additional data on workers, such as their aptitude scores and 
work history. Finally , identification information enables one to 
produce a description of the work being performed by a particular 
person, or to locate by name all individuals who are performing a 
particular task or set of tasks. 

Use of the "Relative Time Spent" Rating Scale 

Now let us consider the rating scale for a few minutes, because 
I believe this to be an Important topic. Research indicated that 
many workers do not have a clear idea of the exact percentage of 
their time devoted to each task they perform. On the other hand. 



they can state with confidence that they spend more time on one i 
task than on another. This led to the development of a "relative 
tlme**spent" scale, by which workers report the amount of work time 
they spend on each task relative to the amount of time they spend 
on other tasks. We use a 7^polnt relative time-spent scale. If 
an Individual does not perform a task he leaves It blank. If he 
does perform It, he rates It from a level "1," which means that he 
spends an extremely small amount of time on It compared to the 
amount of time he spends on other tasks In his job, to a level "7," 
which means that he spends an extremely large amount of time on It 
compared with the amount of time he spends on other tasks in his 
job. These relative time-spent ratings are converted into estimated 
percent time values . The first question often asked by individuals 
reviewing this procedure is "Why percent time? Why not use some 
other factor such as frequency of performance?" 

It is beyond the scope of this paper to discuss all of the 
factors favoring use of percent time-spent estimates, but several 
are sufficiently Important to warrant your consideration. First, 
there are certain statistical characteristics which makes this 
approach extremely useful. It has a clearly defined range with a 
base value of "0," For the individual case, the values Indicate 
the percentage of his work time spent on each task, and the sum of 
these values across all tasks in the inventory is 100%, In a like 
manner, the average values for any group of workers indicate the 
percentage of group time spent on each task, with the sum of these 
values also equalling 100%, Percent time values provide a con- 
venient method for computing the overlap of two individual jobs 
with each other; of an individual job with a group job description; 
or of one group job description with another group job description. 
Results from numerous studies have indicated that matrices reflect- 
ing overlapping time among individual job descriptions, when 
analyzed by the CODAP grouping program, can yield a precise defi- 
nition of the types of jobs existing in an occupational area. 
Finally, having available the percentage of time spent on tasks 
makes it possible to compute the time spent by individuals or groups 
on particular types of work. For example, a manager may wish to 
know how much time is being spent by a group of mechanics on pre- 
ventive maintenance. This can be very quickly computed by the 
CODAP system. It should be noted that none of the above charac- 
teristics apply to a scale such as "frequency of performance," 
How could one possibly summarize the level of activity across a 
series of tasks in terms of frequency, when some of the tasks are 
performed frequently, while other tasks within the subset are per- 
formed infrequently? I strongly recommend use of the relative 
time-spent scale as the primary rating factor in occupational 
surveys, and that the obtained values be transformed into percent 
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tlme**spent estimates. This is a requirement for the CODAP system, 
and it makes possible many types of analyses which cannot be 
accomplished using frequency of perfoinoance data. 

III. THE CODAP ANALYSIS SYSTEM 

By now you have heard me refer several times to CODAP, which 
is the analysis system used not only by the Air Force, but also by 
other military services. There is simply no way in a brief amount 
of time to communicate the power of this system. We have been 
working on it continuously for over 13 years, and the program list- 
ings now run about 1,400 pages in length. It represents an invest- 
ment of hundreds of thousands of dollars, and thousands of in-service 
man hours. Yet, it is without question the most important product 
of the Air Force Occupational Research Project. 

The concept behind CODAP is to provide ways for analyzing, 
organizing, and reporting occupational information so as to answer 
as many management questions as possible, CODAP currently con- 
tains approximately 40 general purpose programs, and several new 
ones are under development. All of these programs are interactive 
and highly efficient. I wish I had time to describe them to you, 
but it would take at least a day to cover them fully. All I can 
do in a few minutes is to mention a few programs which are used 
frequently. (See the Appendix for a description of selected CODAP 
programs and example CODAP outputs) 

Example CODAP Programs 

For example, one program produces a consolidated description 
of the work performed by any specified group of individuals. Such 
a description can be produced for workers at a particular ^>ase; or 
for those who have been in their jobs less than one year; or those 
who claim their talents are not being utilized; or those who work 
on a particular type of equipment — indeed, for any group of workers 
which can be defined in terms of information in the background 
section of the job inventory. A consolidated job description indi- 
cates the percent of group members performitig each task; the average 
percent of work time spent on the task by those who perform it; and 
the percent of group time spent on each task. A CODAP program prints 
the task statements and associated computed values, arranged in terms 
of percent members performing, or in terms of group time-spent 
values. A consolidated description of the work performed by indi- 
viduals during their first year or two on the job is particularly 
useful in validating or designing the curricula for entry-level 
vocational training. 
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Normally when we analyze an occupation i we produce a series 
of job descriptions for groups at various experience levels. That 
Is, we compute consolidated descriptions for Individuals who have 
been In the occupation for less than one year; from one to two 
years; from two to four years; four to eight years; and so on. 
Then the CODAP system Is used to gather this Information Into a 
table which Indicates the percent of Individuals at each experience 
level that perform each task In the Inventory. In this way, we 
find when tasks tend to be assigned, and when training should 
be given In order to be timely. 

Another CODAP program enables managers to study the differences 
In work being performed by any two specified groups of Individuals. 
For example, one might wish to know the differences In work per- 
formed by Individuals at one grade level and those at another grade 
level; or In the work performed by Individuals working on two types 
of equipment. The CODAP system analyzes the two defined groups and 
prints a report summarizing the major differences In work performed. 

Perhaps the most powerful CODAP program is one which identifies 
and describes all the types of jobs which exist in an occupational 
area. Beginning with 2,000 individual job descriptions, this 
program will compute a 4,000,000-element input matrix reflecting 
the similarity of each job with every other job. Then it proceeds 
to group similar jobs into clusters and prints out a description 
of work performed by Individuals in each cluster. The program is 
iterative and may evaluate well over a billion alternative solu- 
tions in arriving at the best definition of job types and clusters 
in a particular occupation. Still another CODAP program can be 
used to determine the characteristics and locations of individuals 
working in each job type and cluster. The results of job typing 
analyses are extremely valuable in identifying changes needed in 
defining occupational categories in an organization or military service- 
Other CODAP programs can be used to compute job descriptions 
for individuals, or for each individual in a specified group, or to 
compute the amount of work time each worker spends on a given set ^ 
of tasks. Using factor ratings in conjunction with task data, 
CODAP can be used to compute the difficulty level or the grade 
requirement for each job. Programs are available within the CODAP 
which will produce two-way frequency distributions between back- 
ground variables; compute the difficulty level of each task; com- 
pute intercorrelations among background variables; determine the 
reliability of task factor ratings; compute the average grade level 
or the average experience level of workers performing each task; 
compute regression equations; print task lists, or print a dictionary 
of background variables. The CODAP system is also a general occu- 
pational information retrieval system. All reports, descriptions. 
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and analysis results computed by CODAP are stored and identified. 
Any subset ^f descriptions or reports can be extracted, ordered, 
and printed. CODAP even numbers the pages in an extracted report 
and automatically prints a table of contents. In general, there is 
a CODAP program available to organize and analyze occupational data 
to answer any question asked by managers of a personnel system. If 
we find that there is another type of analysis which would provide 
information on a question posed by management, then we immediately 
write a new program which will perform the necessary computations. 
This Is one reason why all military services in the United States 
either are, or will shortly be, using the CODAP system for their 
occupational analyses. 

I have probably bored some of you with the details concerning 
the collection, analysis, and reporting of occupational data. What 
you may wish to hear about are some experiences in using the information. 

Adoption of Job Survey Technology by Various Agencies 

In the Air Force we did research on various techniques from 
1958 until 1967. During this period, we collected experimental 
data from over 100,000 cases and developed most of the programs in 
the CODAP system. Although cost savings data were not accumulated 
during this time period, occupational data led to numerous changes in 
training programs and occupational structures. In late 1967 the 
Air Force established an operational unit with 15 persons who 
devoted full time to the construction, administration, and analysis 
of occupational survey data«\ Its mission called for the completion 
of 15 surveys per year. In l\69, the staff of this organization 
was increased to 28, and the mission increased to 24 surveys per 
year. Last year the staff was increased again, to 42 persons, and 
the mission was moved up to 51 surveys per year. Each of these 
increases in staff and mission was due to demonstrated pay-offs of 
occupation information, and to increased demands from managers for 
more timely data. So far, the operational unit has surveyed over 
200,000 enlisted persons in over 150 occupations. At the present 
time 68 surveys are in various stages of completion, and plans have 
already been made for expanding the capability of the unit to meet 
the increased demands for morei occupational data. 

In the Air Force, the greatest pay-off from occupational data 
so far has been in the area of training. Significant changes have 
been made in every training course associated with an occupational 
survey. Frequently these changes have not led to cost savings, 
since they have been in the form of reducing training on certain 
tasks while Increasing it on others. Even so, approximately 
$7,000,000.00 cost avoidance has been documented during the past 
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two years alone, which was directly attributable to reductions in 
training based on occupational survey information. 

Encouraged by the Air Force occupational survey research 
findings, the Marine Corps established an operational unit which 
is currently manned by 37 persons, three of whom work full time 
in maintaining job structures. So far they have surveyed 11 of 
their occupational areas, which contain nearly one-third of their 
manpower. The Marine Corps had the Air Force CODAP system reprogrammed 
to operate on an IBM 360-65 computer? They are particularly happy 
with the job-typing programs, which have produced results leading 
to major changes in the job structures in every occupation surveyed 
thus far. During the past year, they have documented over 
$4,000,000.00 in cost avoidance based upon their occupational 
analysis results. That is a large savings considering the rela- 
tively small size of their personnel system. This year, the 
Marine Corps task analysis group received a Presidential Management 
Improvement Award. 

The Army has an operational job-task analysis group consisting 
of 35 full-time persons. They have been collecting occupational 
data using job inventories for a number of years. To date they 
have been using their own analysis programs, but I understand that 
they are planning several significant changes in their procedures • 
These include the collection of worker identification data; use of 
the relative time-spent factor for a portion of their task list, 
and use of the CODAP to supplement their own analysis system. 

The Navy has recently conducted several large-scale occupational 
surveys using job inventories and process the data with CODAP. The 
Navy officially established an operational job-task group this 
month and is pledged to use the CODAP system for analyses. 

The Coast Guard has been conducting occupational surveys for 
several years with job inventories patterned after those used in 
the Air Force. All of their analyses thus far have been conducted 
using the CODAP system. They have now surveyed about one-third of 
their occupational areas. 

The Canadian Forces have surveyed most of their occupations 
using job inventories, although, to date, they have used their 
own computer analysis programs. The Australian Air Force has 70 
inventories in some stage of development. While these instruments 
tend to include a large number of task statements, they are other- 
wise patterned after those produced by the U. S. Air Force. An 
exchange officer from the Australian Air Force has recently com- 
pleted a 2 1/2 year tour working in the Air Force occupational 
research program and studying the CODAP system. A second exchange 
officer has now moved into this position. 

ER?C 12 



Many Universities, government agencies , and government con- 
tractors have collected occupational data using job inventories, 
and a number of these have accomplished their analyses using the 
CODAP system. To date, the CODAP system has not been available 
to industrial organizations, although it has been used by many 
non-^profit organizations, especially those conducting research 
under government sponsorship. 

I mention all of these programs to emphasize three points. 
First , there seems to be a large movement toward conducting 
occupational surveys using Job inventories; second, many agencies 
are using, or are planning to use the CODAP system for data 
analyses; and third , occupational analysis programs are generally 
in good health and expanding. 

IV. APPLICATIONS OF JOB SURVEY INFORMATION 

So far I have mentioned applications of occupational data In 
the areas of training and work structures. Actually occupational 
data can and should have an impact on almost every part of the 
personnel system. The only thing that is lacking Is the develop- 
ment of appropriate technology, and this is a major mission of 
the Air Force Occupational Research Division. I will have time 
to mention only a few of the applications being studied by this 
organization. 

Task-Level Experience Records 

To my knowledge, no military service and few industrial organi- 
zations maintain individual work experience records at the work- 
task level. In the Air Force a man may be assigned for a two-year 
period as a jet engine mechanic at a particular base and spend 
nearly all of this time working in a shop balancing jet engine 
rotors. When he is transferred to a new base, nbout all we know 
about him is that he spent two years at a particular base as a 
jet engine mechanic. We may assume that he gained experience on 
many maintenance tasks during this period which, in fact, he did 
not perform. We also have no record that he has received so much 
experience in balancing rotors. 

Since a job inventory contains all of the significant tasks 
being performed by personnel in an occupational area, it provides 
an ideal framework for establishing and maintaining individual 
experience records. We know that individuals can be trusted to 
provide an accurate statement of tasks they are currentlj' perform- 
ing by using a job Inventory. It may be that they can also be 
trusted to report their past experiences in terms of tasks listed 
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in the Inventory. This is a topic for research. We see many poten- 
tial applications for task-"level experience records. They can be 
used to evaluate the capabilities of the personnel force, to guide 
on-the-job training programs, to locate individuals with particular 
skills, and to guide reassignments. 

Development of Task and Job Difficulty Indexes 

One recent breakthrough by the occupational research group 
is the development of a technique for evaluating the relative 
difficulty levels of tasks and jobs in an occupational area (Mead, 
1970a; 1970b; Mead & Christal, 1970). Task difficulty is a com- 
plicated concept. A task can be difficult to perform because 
repair parts are hard to get; the technical manual is hard to 
understand; the environment is hot; and several other reasons. 
After considering many alternatives , a definition was selected 
which reflects the amount of time it takes for individuals to 
learn to perform a task adequately. Supervisors in an occupation 
cannot rate the time it takes for workers to learn to perform tasks; 
but they can agree that, other factors held constant, workers can 
learn to perform some tasks faster or slower than they learn to perform 
other tasks. Inter-rater reliability coefficients cif^ceming the 
relative difficulty of tasks within an occupation, based upon 30 
to 40 raters, generally range in the middle to upper 90's. 

We have also developed an equation which enables us to deter- 
mine the relative difficulty levels of jobs within an occupation. 
First, we used the CODAP system to randomly select and publish 
multiple copies of 250 individual job descriptions in one career 
ladder. Then we had supervisors in the field to rank these jobs 
in terms of their relative difficulty levels. Again, we found high 
inter-rater agreement. Using the job rankings as a criterion 
vector, we applied the policy-capturing model (Christal, 1968a; 
1968b; Bottenberg & Christal, 1968) in an attempt to develop an 
equation which would reproduce the policy decisions of the super- 
visors concerning job difficulties. Twenty-two predictors were 
hypothesized; but only three predictors entered into the equation. 
The same approach was repeated in 11 other ladders, and in every 
instance, the same three predictors were able to reproduce the 
supervisory rankings. These results are shown in Table 1. 

Let's study this table for a minute. First notice the three 
predictors. Predictor 1 is the number of tasks in the job, and 
Predictor 2 is the number of tasks squared. Inclusion of the 
squared term is necessary because there is a curvilinear relation- 
ship between the number of tasks in jobs and their perceived 
difficulty levels. Predictor 3 is a complex variable reflecting 
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the average difficulty level of tasks performed per vinit time. 
This variable is computed by the CODAP Variable Generation Program 
(VARGEN), and is simply the cross-products of time spent and task 
difficulty, summed across all tasks in the inventory for a 
particular Job* 

Table L Standaid Score Job Difficulty Equations for 12 Camr Ladders 
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Notice that when these three variables are properly weighted, 
the resulting composite shows remarkable ability to predict the 
supervisory job difficulty rankings. Also notice that the standard 
score weights in all 12 equations look very similar. As a matter 
of fact, they looked so similar that we began to wonder if it might 
not be possible to develop a universal job difficulty equation. In 
order to run a first test of this hypothesis, we averaged the 
standard score weights across the 12 ladders for each predictor; 
then we applied this single general equation back to the 12 samples 
to determine its predictive efficiency. Table 2 presents the results 
from this exercise.^ Notice that in every instance, the general 
equation is almost as efficient as the least squares equation 
developed within the sample. 

All of the career ladders shown in Tables 1 and 2 are rela- 
tively low level, and there is some question as to whether the 
general equation will be applicable to more technical occupations. 
In one recent study the general equation yielded a valid coefficient 
of ,89 for the Information career ladder, which is moderately 
encouraging, and ,80 for the Weather Observer career ladder » 



^Only 11 ladders are shown; computations for the remaining 
ladder have, not been completed. 
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Table 2, Effictency of the Job Difficulty General Equation 
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which is not so encouraging. If it turns out that the general 
equation holds up, we will no longer have to go to the trouble of 
establishing criterion vectors. We will simply have 30 to 40 super- 
visors in each area rate the task difficulties; then we can apply 
the general equation to determine the difficulty levels of all 
jobs surveyed. We already have programs in the CODA? system for 
performing the necessary computations. 

Now I will discuss a few of the applications we have made of 
task and job difficulty indexes. 

Technical School Versus On-The - Job Training 

In over 30 career ladders^ the Air Force has established what 
it calls a Category B training plan. Under this plan, a proportion 
of the assignees to the ladder are sent directly to the field with 
no formal training, while the remaining individuals go through an 
entry- level resident training course, and then are sent to the field. 
One would think that, if there is any flexibility in work assignment, 
those who have had the benefit of formal resident training would 
tend to inherit the more complex and difficult jobs. We have tested 
this hypothesis, using the job difficulty index as one criterion. 
The analysis is a model-seeking design, with two treatments and a 
concomitant variable. The two treatments are Technical School 
Graduates (TSG) and Directed Duty Assignees (DDA) ; the concomitant 
variable is Time in Military Service (TMS) ; the criterion is job 
difficulty level; and aptitude Is held constant. Figure 1 shows 
some of the possible models. 
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Tinw In ScrviM Time In Service 



Fig. I, Comparison of technical school graduates and directed duty 
assignees m terms of difficulty level of woik inherited. 

Model A shows an unlikely situation where the difficulty of work 
does not change over time, and where the regression lines for 
TSGs and DDAs are identical. Model B shows job difficulty increas- 
ing over time, but again, the regression lines for the two groups 
are identical. Model C shows the difficulty level of work increas- 
ing over time, and the regression lines to' be parallel: but at all 
points in time, the TSGs are inheriting more complicated jobs than 
the DDAs. Model D was considered to be the most likely. It shows 
the difficulty level of work increasing for both groups over time. 
However, during the early time period, the TSGs are being assigned 
less difficult work than the DDAs. This makes sense, since the 
DDAs will have been on the job while the TSGs were in the class- 
room. However, Model D shows the TSGs catching tip with and passing 
the DDAs in terms of the difficulty level of work inherited. The 
basic question is when does this cross-over occur. With a re- 
enlistment rate of 20 or 30 percent, it needs to occur well before 
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the end of the first enlistment If the technical training course 
is to be worth its cost. The analysis program locates the appro- 
priate model and reports all of the parameter values. 

Detailed results from application of the model to eight career 
ladders are reported by Lecznar (1972). In general, Model B was 
found to be the correct representation. That is, while the 
difficulty levels of jobs increased over time, at no point (between 
the 5th and 35th month of service) was one group revealed to per- 
form more difficult jobs than the other. The same type of analysis 
was run against five other criteria: (a) number of tasks performed, 
(b^) average difficulty level of tasks performed per unit time, (c,) 
official airman performance ratings, (d^) job interest, and (e) felt 
utilization of training and talents. 

With a few minor exceptions, no differences were found between 
TSGs and ODAs in any of these analyses. Figure 2 displays results 
for the Cononunications Center Career Ladder (291X0) . The regression 
lines are identical for TSGs and DDAs against all criteria except 
for the number of tasks performed. There is a slight tendency for 
those trained on the job to perform more tasks than those trained 
in the formal entry-level course. 

^ There has been a concerted effort in the Air Force to make 

On-the-Job Training (OJT) programs equivalent to resident training 
programs for the Category 6 areas. This study provides more evi- 
dence that equivalency is being attained. As an interesting side- 
light, a recent study has been completed comparing the relative 
costs of resident training versus OJT for the 291X0 Career Ladder 
(Dunham, 1972). The results of this study indicated the cost of 
technical school training to be $2,780^ while the cost of OJT was 
estimated to be $1,311 (median cost estimate). The upper limit of 
OJT cost estimates (95% confidence) was reported to be $1,515, 
which is still considerably less than the reported cost of resi- 
dent training. Most of this difference is attributable to the 
costs of equipment, maintenance, training aids, and administration, 
which are calculated to be considerably less for on-site training. 

Difficulty of Work Assigned to Personnel as a^ Function of 
Their Aptitude Levels 

In 1967 the Department of Defense established a program titled 
"Project 100,000" which required the military services to accept 
quotas of personnel who fell below previously operating mental 
standards. The Air Force initiated a number of studies to deter- 
mine how these individuals succeeded during their first enlistment 
(four years) • Data were easily obtained concerning elimination 
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rates in basic training, technical schools, and during post-school 
service. Scores were also available on written proficiency tests 
and official performance rating forms. However, there was one 
matter which plagued the investigators: "How do we know that these 
individuals are not being assigned tasks to perform as a function 
of their aptitude levels?" True, aptitude scores were not being 
provided to supervisory personnel. Hovever, a supervisor might 
perceive that some of his subordinates are more talented than 
others, and he might assign the brighter airmen to do the compli- 
cated tasks, while assigning tasks such as sweeping the floor to 
airmen he perceives as being "dull." And if a "dullard" does an 
excellent job of sweeping the floor (his assigned task), and if 
he does it with a good attitude and with enthusiasm, will the 
supervisor give him a bad performance rating? Answer: "Probably 
not." Since the Air Force can use only so many floor sweepers in 
a specialty, it was deemed important not to draw any conclusions 
about the success of "New Mental Standards" (NMS) airmen until it 
was determined whether they had been given differential treatment 
as a function of their ability. 

Since the definition of New Mental Standards airmen changed 
several times during the course of Project 100,000, the question 
was generalized to evaluate the difficulty of work assigned 
personnel as a function of their aptitude levels (Wiley, 1972). 
Data were collected using job inventories from approximately 
14,000 airmen in 11 career ladders which had received large inputs 
of low aptitude personnel. In fact, approximately 47% of the 
cases (6,520) were Category IV personnel, while nearly 27% of the 
cases were classified as New Mental Standards airmen.^ 

The data were analyzed using the Multiple Linear Regression 
Model. In general, the approach was to determine the unique con- 
tribution of aptitude in accounting for the variance In (a) job 
difficulty, (b^) the number of tasks performed, and (c^) the Average 
Task Difficulty Per Unit Time (ATDPUT). Variables such as in- 
service training, time-in-service, time-in-job, grade, job location, 
coinnand assignment, etc., were held constant. Results of these 
analyses are summarized in Table 3. 



ERIC 



^As an interesting side issue, a record was kept of the num- 
ber of booklets which had to be rejected from processing because 
of failure to follow instructions by the incumbents. Only about 
2% of the booklets were rejected from the NMS subsample, which is. 
evidence that even these personnel could read and provide the 
required data in an occupational survey conducted with job inven- 
tories. Nothing was revealed during the analysis phase which 
caused the Investigators to question the quality of Information 
provided by this subsample. 
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TABLE 3. UNIQUE CONTRIBUTION OF APTITUDE (AFQT) IN PREDICTING 
DIFFICULTY OF WORK ASSIGNED 
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^Average Task Difficulty Per Unit Time 
^Sign. at .01 level 
^Sign. at .05 level 

Although several of the R^s in Table 3 are significant at the 
•05 or .01 levels I in no instance did the unique contribution of 
aptitude exceed .02 • It was concluded that there was no practical 
difference in the difficulty level of work being assigned to per- 
sonnel in these 11 ladders as a function of their aptitude levels. 
Various interpretations can be placed upon this finding. It could 
be that supervisors in these ladders have very little flexibility 
in the way that they can assign work to subordinates. Another, 
and possibly more straight-forward interpretation, is that 
personnel assigned to these ladders have sufficient aptitude 
to perform all available tasks. In any event, it appears that 
the NMS airmen did not receive differential treatment with respect 
to their work assignments. 

One unexpected finding was that, in nearly all Air Force 
Specialties (AFSs), aptitude correlated higher with the ATDPUT 
variable than it did with the job difficulty composite. A tendency 
was observed in several ladders for supervisors to assign fewer, 
but more complex tasks, to^'the brighter personnel. The signifi- 
cance of this finding will be discussed later. ^ 
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Differences in Work Assigned to Blacks and Non-Blacks 

Since approximately 19% of the cases in the NMS airman study 
described abova were Black, the data provided an ideal base analyzing 
racial differences on variables such as the difficulty of work 
assigned. Job interest, and felt utilization (Christal, 1972). 
Table 4 presents zero-order correlations between Race (Black = 1; 
0 otherwise) and selected variables within each of the 11 career 
ladders in the MS study. Although some of the relationships 
reported ia this table are significant, they are all very low. The 
more meaningful findings are reported in Table 5, which displays 
the unique contribution of Race in the prediction of the job 
assignment-satisfaction criteria. The variables held constant 
in each equation are presented in Table 6, 

Three of the criteria are associated with the nature of work 
being performed by incumbents in the various career ladders: (a) 
the number of tasks being performed, (h) the average task difficulty 
per unit time, and (c) an index of job difficulty. As indicated in 
Table 6, the variables held constant related to age, training, 
aptitude, education, and experience. When these variables were 
held constant, it was found that there were no significant differ- 
ences in the number of tasks being assigned to Blacks and Non-Blacks 
in the samples tmder consideration. Furthermore, there were no 
significant differences in the average difficulty levels of tasks 
performed, weighted by the time spent on each task. However, when 
these two criteria were weighted into an index of overall difficulty 
level, it was found that Blacks were being assigned significantly 
less difficult jobs in two career ladders: 605X0 Air Passenger/ 
Air Cargo and 702X0 Administrative. Although these differences 
were statistically significant at the ,01 level, they were, 
nevertheless, small. In each instance, the race variable uniquely 
accounted for less than 1% of the criterion variance. 

Table 5 also reflects racial differences in expressed job 
interest and in reported utilization of talents and training. Sig- 
nificant racial differences appeared in only two career ladders. 
In each instance, however, they were in the direction that sug- 
gested the Blacks found their jobs irore interesting and felt that 
their talents and training were being better utilized than did the 
Non-Blacks. These findings are unusual in two respects. First, 
in the case of the 291X0 Communications Center Ladder, the unique 
contribution of race in accounting for feelings of being well 
utilized had an F ratio of 27.48, which is highly significant. 
Even though the Blacks and Non-Blacks were being assigned jobs 
and tasks of comparable difficulty levels in this ladder, the 
Blacks felt that they were being better utilized. In the case 
of the 702X0 Administrative Career Ladder, it was found that the 
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Table 6. Predictors Used to Account for Variance in Selected Criteria 



Crittrion 



Pradlctor 


Job 
Indtx 


NHmbtr 
of TliHs 
Mrformtd 


A¥9 Task 
OifflCHlty 

Ptr 
Unit TImt 


Job 
lnt«r«st 


F<lt 
Utilixation 
of Taltnts 

and 
Training 


Months in Job 


X 


X 


X 


X 


\ 


Months in Career Ladder 


X 


X 


X 


X 


X 


ioi:iI Mo:vJ>s AcUve Military Service 


X 


X 


X 


> 


A 


Vv:j.r% of l:cji:.:2tion 


X 


X 


X 


A 


\ 


.'i . Cci'iiilc 


X 


X 


X 


X 


X 


AC.i{i Mccriunical Al 


X 


X 


X 


X 


l\ 


aQl AQtraaistrative Al 


X 


X 


X 


X 


X 


aQE Gcnerai Al 


X 


X 


X 


>L 


X 


aQE Hlec cronies Al 


X 


X 


X 


X 


X 


Technical School Graduation (Yes/No) 


X 


X 


X 


X 


X 


Age at tinlistment 


X 


X 


X 


X 


X 


Job Difficuliy Index 








X 


X 


Number of Tasks Performed 








X 


X 


Average Task Difficulty Per Unit Time 








X 


X 










X 


X 


Number of Subordinates 








X 


X 


CONUS Assignment 








X 


X 



V 



Blacks were being assigned jobs which were slightly less diffi- 
cult than jobs assigned the Non-Blacks. In spite of this, the 
Blacks expressed a higher feeling of utilization and job interest 
than did the Non-Blacks. In the remaining nine career ladders, 
there were no significant differences in expressed attitudes. 

Prediction of Civilian Grade Classification and Analysis 
of Biases in Classification Actions 

Until recently. Air Force experience in conducting occupa- 
tional surveys with job inventories was restricted to enlisted 
and officer samples. Analyses revealed survey data to be of 
good quality. In 1971 a first attempt was made to survey Civil 
Service personnel (Garza, 1972). Since civilian grades are 
tied to job content, there was some fear that a civilian might 
feel he had something to gain by being dishonest, or something 
to lose by being honest in describing his job. Fortunately, 
there was no indication of distortions in the data received from 
5,485 job incumbents surveyed in seven series of the Accounting 
and Finance occupational area. Several analyses were made of 
these data (Carpenter & Christal, 1972) which utilized task and 
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job difficulty Indexes. One Involved an attempt to predict the 
official grade classification of each position, and the second 
Involved an analysis to detect biases In grade classification 
actions. The reader should understand that grade classification 
actions are based upon job descriptions provided to the classi- 
fier by the supervisor, while the analyses reported below are based 
entirely upon data reported by job Incumbents in job Inventory 
forms. 

Table 7 indicates multiple Rs which were obtained from infor- 
mation concerning the amount of time spent by Incumbents on par- 
ticular tasks. In order to reduce the problem of overflttlng, 
the tasks were split into two problems, and those entering either 
equation were used as predictors in a third problem. Although 
this did not completely eliminate capitalization on chance, it 
did reduce it to tolerable limits, considering the number of 
criterion observations available. 





TABLE 7. 


CORRELATION 


OF TASK VARIABLES WITH 


AUTHORIZED GS GRADE 








Group A 


Group 


B 


Composite 






# VARs 


Mult. 


# VARs 


Mult. 


# VARs 


Mult. 


GS Series 


N 


Entering 


R 


Entering 


R 


Entering 


R 


501 


856 


68 


.741 


73 


.755 


77 


.813 


520 


1,305 


55 


.564 


49 


.582 


76 


.649 


525 


1,710 


66 


.720 


69 


.691 


81 


.755 


530 


203 


39 


.869 


36 


.896 


38 


.924 


540 


307 


49 


.724 


49 


.752 


65 


.819 


544 


604 


29 


.725 


32 


.759 


44 


.793 


545 


500 


41 


.772 


48 


.752 


57 


.805 



The task variables which entered the composite model were 
then combined with certain background variables to predict 
authorized GS grade. These background variables included those 
relating to the difficulty level of the job (Number of tasks, 
ATDi^UT, and Job Difficulty Index); job location; command; personal 
characteristics of the incumbent (such as sex, age, marital status, 
etc.); and a large number of incumbent experiences and training 
variables. Readers who wish detailed information concerning the 
subsequent analyses can refer to the published report (Carpenter & 
Chrlstal, 1972). A series of full and restricted regression models 
were computed to determine whether non-job-related variables had 
influenced grade classification actions. In general, it was 
found that factors such as the inctmibent*s age, sex, marital 
status, and factors such as command assignment and job location, 
did not systematically demonstrate a significant source of bias 
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in grade determinations. One exception was the discovery that 
jobs in the Washington DC area are classified about one-half 
grade level higher than Jobs located elsewhere, other factors 
held constant. 

Table 8 presents the zero validity coefficients of the Number 
of Tasks Performed, ATDPUT, and the Job Difficulty Index (computed 
with the general equation) for the authorized GS grade levels of 
jobs in the seven series. Except in the 520 and 540 series, the 
ATDPUT variable showed substantial validity for all series. One 
discouraging finding was that in three ladders, the job difficulty 
equation, as computed by the general equation, had less validity 
than one of the individual variables weighted into that equation. 
Although the general equation may universally predict supervisor's 
ratings of job difficulty, it would appear wise to retain the separ- 
ate elements of the equation in attempting to predict other criteria. 

TABLE 8. VALIDITY OF CERTAIN MEASURES OF JOB DIFFICULTY FOR 
PREDICTING AUTHORIZED GS GRADE CLASSIFICATION 



GS Series 



Predictor 


501 


520 


525 


530 


540 


544 


545 


ATDPUT^ 


.51 


.17 


.54 


.47 


.23 


.58 


.51 


No. of Tasks Performed 


.25 


.20 


.23 


.65 


.18 


.35 


.32 


Job Difficulty Index 


.45 


.30 


.39 


.67 


.29 


.46 


.51 



^Average Task Difficulty Per Unit Time 

Task Factor Approach to Job Evaluation 

Another potentially important use of occupational data is 
in the area of job evaluation — the process by which pay and grade 
requirements are associated wi<:h jobs. Although there are a number 
of approaches to job evaluation, most large agencies use some type 
of point rating system. In such a system, each job is rated on a 
series of job evaluation factors, and the factor ratings are 
weighted into a job evaluation composite score. The job evalua- 
tion composite score Us, in turn, converted into a grade or pay 
requirements level. The factors and factor weights employed in 
many job evaluation systems are in reality regressions equations 
developed to predict agreed-upon grade or pay levels for a set 
of benchmark jobs. 
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Although development of a job evaluation system Is relatively 
straight-forward, applying it is another matter. How does one 
obtain unbiased factor ratings on tens or hundreds of thousands 
of jobs? We are working on a system using task-level data which 
may be of interest to you. Instead of obtaining factor ratings 
on jobs, we are obtaining them on tasks which have been included 
in job inventories • Since every job in an occupation can be 
^escribed in terms of the tasks in an inventory, we can simply 
apply CODAP and compute the factor scores for each job. Also, 
CODAP can be applied to weight the factor ratings for each job 
into a job evaluation composite. This process assures that job 
evaluation is conducted in a systematic and unbiased manner. 

In one recent test of this technology, an attempt was made to 
develop a system for evaluating the skill-level requirements for 
jobs in two Air Force career ladders. Task-level job descriptions 
were published using the CODAP system for 400 positions in the 
Fuel Services ladder (631XX) and 677 positions in the Personnel 
career ladder (732XX) . These descriptions were rated for appro- 
priate skill-level requirements by senior supervisors in the 
field, yielding mean ratings with computed inter-l*ater reliability 
coefficients of .95 and .92, respectively. 

An independent group of supervisors in each ladder rated all 
tasks in a job inventory for their ladder on a series of job eval- 
uation factors, such as knowledge requirements; responsibility 
for use and control of money, supplies, or equipment; required 
special training and work experience; oral and written communi- 
cations; supervisory responsibility; decision -making requirements; 
task difficulty; etc. All such ratings were collected using an 
8-point relative task comparison scale. The instructions went 
something like this: "Compared v/ith other tasks in your career 
ladder, when a man is performing this task» what level of communi- 
cations skill is he exercising?" If no communication skills are 
required, the task is assigned a value of **0.** If communication 
skills are involved in performing the task» a rating of from 1 to 
7 is assigned, according to the level of requirement compared 
with other tasks in the ladder. Reliability coefficients for 
the various factors (based on vectors of mean ratings) ranged 
from .93 to .99. 

Scores were developed on each job description for each factor 
using the CODAP system. For example, one variable reflected "the 
average level of communication skills exercised per unit time.'* 
For a particular case, this is simply a sum of the cross-products 
of the time spent values and task indexes, across all tasks in 
the inventory. The factor information was then combined with 
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other job information, such as the number of subordinates super- 
vised, in an attempt to predict the supervisory skill-level 
criterion ratings. 

It was found that a major portion of the criterion variance 
could be accounted for by only four variables plus two squared \ 
terms, as follows: 

1. Number of tasks performed 

2. Number of tasks performed, squared 

3. Number of subordinates 

4. Number of subordinates, squared 

5. ATDPUT (Average Task Difficulty Per Unit Time) 

6. ADMPUT (Average Decision-Making Requirement Per Unit Time) 



TABLE 9. PREDICTION OF SKILL LEVEL REQUIREMENTS FOR 
JOBS IN TWO AIR FORCE CAREER LADDERS^ 

N R R . 

Career Ladder Jobs Full Model 6-Predictor Model 



Fuel Services 400 .96 .95 

Personnel 677 .94 .92 



^See text for definition of predictors. 

Table 9 reports the Multiple Correlation Coefficients for the 
full model (33 predictors) and for the six variables previously 
shown. It should be noted that application of the six-variable 
equations require obtaining task ratings from supervisors on only 
two factors: Task Difficulty and Decision-Making Requirement. The 
number of tasks performed by each worker is automatically computed 
by CODAP, and the number of subordinates is normally obtained by a 
background question included in job inventories. The two squared 
terms in the equation were required in order to account for anti- 
cipated curvilinear relationships. The number of tasks in jobs 
normally increases from the apprentice level through the first-line 
supervisory level; then falls off somewhat at the superintendent 
level. This is because a first-line supervisor both supervises and 
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performs tasks, while a superintendent normally does not perform 
journeyman-level tasks* In a like manner, the number of Immediate 
subordinates reaches a peak at the first-line supervisor level. 
Note all three variables which enter Into the Job Difficulty 
Equation entered Into the skill-level equation for the two ladders* 

Use of Task Difficulty Indexes in Determining Training Requirements 

In the Air Force we have saved millions of dollars by ellmlna* 
ting training on tasks which occupational survey data have revealed 
to be performed by few workers. However, data concerning the proba«- 
blllty of school graduates doing certain tasks In their first or 
second assignment Is insufficient to make sound decisions concern- 
ing school curricula (Chrlstal, 1970). We are attempting to collect 
additional information on each task in an inventory in order to 
determine how much emphasis should be given to it in the entry-level 
course. These factors give consideration to such things as the 
consequences of inadequate performance; the probability that the 
task can be performed adequately without specialized training; 
the estimated cost of teaching the task in a formal training 
course as compared with teaching it on the job; the probability 
that the task may have to be performed in an emergency condition, 
where there is no time to obtain information on how it should be 
done; the perishability of the skill; and so on. The method for 
combining such information into curriculum decisions is not simple. 
For example, ordinarily a task will not be included in a course if 
the probability of a worker having to perform it is very low. On 
the other hand, if the task might have to be accoiiq>lished in an 
emergency, such as performing mouth-to-mouth resuscitation, or 
fighting a certain type of chemical fire, or evaluating a radia- 
tion count — and if the consequences of inadequate performance 
are serious, and if the probability of being able to perform the 
task adequately without formal training is low — then it certainly 
should be included in the entry-level course. Our goal is to 
develop procedures for evaluating each task on factors such as 
those I have described, and to develop equations which can be 
applied to each task to determine how much emphasis should be 
given to it in the training course. 

Data on factors such as those previously mentioned have been 
collected on tasks in the Medical Services Career Ladder. Analyses 
revealed that correlation between Task Difficulty Indexes and the 
rater "probability of adequate performance without specialized 
training" was .96. Thus, once again. Task Difficulty Indexes, 
where difficulty is defined in terms of relative time to learn, 
will assist in solving an operational problem. 
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Before leaving this topic, I would like to make two points 
concerning the application of occupational data to training. 
First s one simply cannot provide massive computer printouts of 
occupational Information and expect trainers to use them. He 
will have to explain how the data can be used and also convince 
trainers that the data are trustworthy. In some Instances you 
will find trainers who are reluctant to eliminate training or 
tasks, even though there is overpowering evidence that such a 
move would be cost effective. In the Air Force, directives have 
been issued which require trainers to review survey data and to 
'*show cause" as to why tasks performed by few workers should be 
included in the entry-level course. Second, a course curriculum 
cannot be designed solely from job survey data. Once the tasks 
to be trained have been identified, trainers must go into the field 
to observe the number and sequence of steps required in the perfor- 
mance of each task to be included in the course. 

Determination of Relative Aptitude Requirements Using a^ Benchmark Seal 

Aptitude requirement levels for entry into various Air Force 
career ladders were initially established by judgment. Adjustments 
have been made from time to time, based on recommendations from 
field commanders and trainers. When an aptitude requirement has 
been raised, it has generally been because someone has gone to a 
great deal of trouble documenting the fact that the jobs in an 
occupational field have become more complex and demanding; Require- 
ments for new career ladders have been set by comparing them with 
similar existing career ladders. The subtests entering into 
aptitude composites have routinely been validated against train- 
ing course grades; but only in a few Instances have attempts been 
made to correlate subtests or composites with actual performance 
on the job. This has been due mainly to the non-existance of 
acceptable job performance measures. 

During the past 25 years I can remember many times when 
Hq USAF asked us to validate aptitude requirements for all career 
ladders. In each Instance, we had to provide soft answers. The 
fact of the matter is that, at the present time, it is not 
possible to defend any level of aptitude as being an absolute 
minimum for success in a career ladder. If training is carefully 
designed and Increased, individuals with lower aptitude might 
succeed who otherwise would not. Job performance aids and 
automatic checkout equipment can reduce the requirements for 
high aptitude personnel in some ladders. Given sufficient time 
on the job, low aptitude personnel can learn to perform many 
tasks which they could not learn in a short period of time. There 
are many such factors that have trade-off values with aptitude. 
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Even if factors such as trainings experience, and performance aids 
are held constant » there is still no completely satisfactory way 
to establish aptitude minimums in the absence of objective work 
performance standards and a method of evaluating personnel against 
such standards. 

Yet» the matter of setting realistic aptitude requirements 
can be extremely important • For example, lowering a requirement 
from the 80th to the 60th centile could double the ntimber of 
individuals eligible to volunteer for service in a particular 
career ladder. As we have moved into a zero*-draft environment, 
there has already been some degradation of high aptitude talent 
in the applicant pool. We may reach a point where it is no 
longer possible to fill quotas at stated requirement levels. 
What then? 

One solution would be to generate more applicants through 
offerings of increased pay and benefits. However, predictions 
of lower budgets and expenditures on national defense does not 
make this approach look promising. It may become necessary to 
find some way of operating th^ force with less talent. This being 
the case, now is the time to begin building contingency plans for 
actions which could be taken if we run into trouble. 

During th^ course of some of the studies described earlier 
in this paper, several members of the occupational research team 
developed a strong feeling that task and job difficulty, defined 
In terms of time to learn, reflects aptitude requirements. Some 
support for this hypothesis can be found from the results of train** 
Ing research. If training time Is held constant, then aptitude 
relates to the amount of material mastered; but If everyone is 
trained to a standard and allowed to move at his own pace, then 
aptitude relates to training time. Another clue came from a study 
in which we computed a correlation of .92 between supervisory 
ratings of task difficulty and the proportions of an independent 
group of supervisors agreeing that tasks could be performed 
adequately by low ability workers. 

Following these clues, we initiated a series of systematic 
studies to determine the relationship between relative aptitude 
requirements for work tasks as judged by behavioral scientists, and 
relative difficulty levels for the same tasks as judged by super- 
visors (Fugill, 1972; 1973). In the first study, a correlation of 
.89 was obtained for a set of mechanical tasks. In a fcllow->on 
study, a correlation of .93 was obtained for a set of electronic 
tasks. Finally, a correlation of .89 was obtained for a set of 
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administrative tasks. Thus, a close correspondence between diffi- 
culty defined in terms of "time to learn" and aptitude requirements 
was established. 

These findinjgs suggest that one can use relative difficulty 
ratings from supervisors to compare the relative aptitude require- 
ments for tasks and jobs within career ladders. This information 
alone can be of some benefit for building contingency plans. For 
example, most career ladders contain several types of jobs which 
may vary in difficulty. The CODAP analysis system can be used to 
identify these job types, and difficulty indexes can be used to 
determine which job types might be shredded out into new management 
units for performance by lower aptitude personnel. The task diffi- 
culty indexes can also be used to identify tasks which might be 
pulled out of existing jobs and engineered into new jobs for per- 
formance by less talented individuals. However, in order to build 
the most meaningful contingency plans, what is needed is a method 
for comparing aptitude requirement levels for jobs across all career 
ladders. This would help the Air Force determine where stated 
requirements can be reduced with the least danger to mission accom- 
plishment, and without changes in the classification structures. 
The approach taken to solve this problem was the development and 
application of benchmark scales. 

Now, let me outline this approach in general terms. 

Step !• Select a set of career ladders requiring the same 
type of aptitudes, for which job inventories and recent occupational 
survey data are available. 

Step 2. Collect ratings from supervisors to determine the 
relative difficulty levels of all tasks within each ladder. 

Step 3. Select 30 to 40 tasks at various difficulty levels 
from each ladder. This will form the benchmark set. Reliability 
of final results will be enhanced if the tasks selected for the 
benchmark set are v/ell known or easily observed. 

Step 4^. Obtain relative aptitude requirement ratings for 
tasks in the benchmark set from knowledgeable behavioral scientists. 

Step 5. Within each ladder, compute least squares regression 
equations to predict task aptitude requirements form task difficulty 
levels. 

Step 6^. Apply the equations developed in Step 5 to rescale 
all tasks in all ladders into a coiranon aptitude requirements frame- 
work (the benchmark scale). 
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Figure 3. Relative Aptitude Requirement Within Benchmark Set^ 



I*d better stop at this point to make sure that these last 
two steps are tmderstood. Figure 3 presents 20 points represen- 
ting 20 tasks on a particular career ladder Which vere Included 
in the benchmark set. The position of a task on the vertical axis 
represents its difficulty level relative to all other tasks in its 
own career ladder. Its position on the horizontal axis represents 
its aptitude requirement level relative to other tasks in the 
benchmark set of tasks. A line of best fit has been drawn through 
the points. Using this graphs the relative difficulty index 
values can be converted into aptitude requirement levels for 
all tasks in the career ladder. If this procedure is repeated for 
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all ladders having tasks represented in the benchmark set, the 
final product is a set of values indicating the relative aptitude 
requirement levels for all tasks in all ladders. The same results 
can be obtained, without actually plotting points and reading 
graphs, by simply developing and applying least-squares regression 
formulae. 

Step Using the task aptitude requirement data, the CODAP 
system is applied to occupational survey data to determine the 
relative aptitude requirement levels fot all jobs in all ladders. 

Step 6. The requirement levels for 1st term jobs are compared 
across ladders. 

Step 9^. The requirements levels are determined for each type 

of job identified in each career ladder by the CODAP system. 

Step 10 . The amount of work time being spent on low requirement 
tasks is determined for each job in every career ladder. 

A test application of this technology was made by the Air Force 
Human Resources Laboratory's Occupational Research Division. Part 
of this study was accomplished in-house, and partly by contract with 
the Systems Development Corporation. I won't have time to report 
all of the details of the study, but I will report generally how 
It was conducted. 

In the first stage, an in-house study was completed which 
involved 10 ladders in the Administrative and General aptitude 
areas. Two hundred and seven tasks were selected for inclusion in ^ 
the benchmark scale, which represented the range of difficulty in 
each ladder. As a second consideration, the tasks selected were 
those which in-service personnel were most likely to understand 
without observation in the field. Twelve in-service behavior 
scientists rated tasks in the benchmark set on relative aptitude 
requirements, and 40 to 100 supervisors in the field rated all tasks 
within their respective career ladders on a relative difficulty 
scale. Correlations between task difficulty levels and aptitude 
requirement levels were generally in the upper 80 's for the bench- 
mark tasks representing various ladders. Regression equations were 
computed and applied to rescale all tasks in all ladders in terms 
of their relative aptitude requirement levels. 

Realizing the dangers of having in-service personnel rate 
tasks without first-hand knowledge, a contract was negotiated with 
Systems Development Corporation to repeat the study on 10 career 
ladders, six of which were identical to those in the in-house study. 
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The contractor study involved 280 benchmark tasks. Six behavioral 
scientists spent six weeks at several bases observing these 280 
tasks being performed and interviewing workers and supervisors before 
executing their independent ratings of relative aptitude requirements. 

It turned out that, for the six common career ladders, the con- 
tractor and in-house studies produced essentially equivalent results. 
Therefore, the two studies were merged, yielding relative aptitude 
requirement levels for all tasks in 14 career ladders. These apti- 
tude requirement indexes were applied to occupational survey data 
in the 14 career ladders, and relative aptitude requirement levels 
were computed for every position in each of these ladders. For 
reasons which are too complicated and numerous to discuss in this 
presentation, the individual position requirements were based upon 
the "average requirement level of tasks performed per unit time." 
The CODAP system was then used to compute the mean, distributions, 
and standard deviations for first-termer jobs in each ladder, as 
well as for each job-type in each ladder. 

The primary results of the completed study are shown in Figure 4. 
Remember that the data in Figure 4 gives consideration to the diffi- 
culty level of every task in thousands of individual first-termer 
jobs. For each ladder, the horizontal line reflects the relative 
aptitude requirement levels for first-term jobs falling minus one 
and plus one Standard Deviation around the Mean. That is, the 
bars show the aptitude requirement levels for approximately the 
middle 68% of jobs in each ladder. The left hand column in this 
Figure indicates the current aptitude requirement levels for the 
14 ladders. The vertical lines, representing estimated AQE equiva- 
lents, have been di.awn arbitrarily, and are designed to assist in 
evaluating the relative aptitude requirements. 

If the Air Force cannot fill quotas at the 80th centile level, 
it would appear that requirements could be lowered to .70 in both 
the Information and Weather career ladders. Under more severe 
pressures, the Accounting and Disbursement requirements could be 
dropped from 80 to 70. The actual aptitude requirements for all 
four of these ladders appear to be less than they are for the 
Procurement ladder; yet the current stated requirement for the 
Procurement ladder is only at the 70th centile level. 

At the 60 level, the Air Force should be able to drop the 
requirement for the Communications Center ladder to 50 and the 
Medical Material Ladder all the way down to 40. In the face of 
drastic shortages, consideration could be given to dropping the 
requirement for the Administrative Ladder to 30. Even if we do 
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not run into problems in filling quotas, the data in this figure 
could be used to make certain adjustments which would bring ladders 
into proper alignment* 

Most career ladders in the Air Force contain more than one 
type of job. For example in the Disbursement Accounting ladder, 
some individuals spend full time computing travel vouchers, some 
keeping manual military pay records; some operating military pay 
computers; some paying and collecting cash; and so on. The CODAP 
system includes programs for identifying and defining the job types 
in each ladder. One type of contingency plan which could be Imple- 
mented in an emergency is to shred out certain job types within 
ladders which can be performed by individuals with lower aptitudes. 
A few examples are shown ir Figures 5, 6, and 7* 

So far I have described two types of contingency plans. One 
involved identifying career ladders for which the aptitude require- 
ments can be lowered with the least danger to mission accomplish- 
ment. The second involved identifying job types within existing 
ladders which can be separated out and made into new specialties 
for performance by individuals with less talent, A third type of 
plan involves removing simple tasks from existing jobs and 
engineering them into new jobs for performance by individuals 
with less talent. While I will not discuss this third alter- 
native today, I would like to mention that the CODAP analysis 
system will compute the amount of work time being spent by every 
individual in a career ladder on such easy tasks. Other CODAP 
programs can provide summary tables indicating the amount of time 
spent on low-level tasks by all individuals at various locations. 
If one proposed to re-engineer jobs, this would tell him where he 
might have the best chance of success. 

Job Satisfaction Research 

There is a great deal of research evidence in the civilian 
sector indicating that factors related to job content and job con- 
ditions influence the decisions of individuals to stay with or 
leave work situations. In the Air Force, factors related to job 
content, assignment location, and worker-supervisor interactions 
are among those frequently cited by individuals for their decisions 
to leave the service. As we have moved into a zero-draft environ- 
ment, retention of qualified workers has become an extremely 
Important goal. Such individuals are available in limited quan- 
tities, and they are difficult to enlist, expensive to train, and 
hard to replace. In recognition of this, the Air Force has 
recently placed increased emphasis on job satisfaction research, 
A full-time effort in this area was initiated a little over a year 
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ago. Fortunately, data on two factors (**Job Interest" and 
"'Utilization of Talents and Training") had already been collected 
In job Inventories for over 130,000 workers In approximately 150 
occupational areas. Detailed analyses of data on these two factors 
are currently undexvay, but a few observations have already been 
made (Gould, 1972). Extensive differences In expressed job satis- 
faction have been found to exist between career ladders and among 
Individuals within career ladders. For example, in some ladders 
fewer than 5% of the workers report that their talents and train- 
ing are not being utilized in their present work assignment; while 
in other career ladders, over 50Z of the workers report their talents 
and training are being utilized "very little" or "not at all." We 
have conducted Intensive studies in a few career ladders in which a 
large number of individuals report low interest and utilization, and 
we find that in many instances there is ample justification for such 
feelings (Stacy, 1973). 

So far, w^e have been able to account only for a modest portion 
of the variance in attitudes among Individuals within career ladders. 
Tables 10 and 11 report the validities of certain predictors for the 
Interest and utilization factors. The full model includes predictors 
such as job difficulty, grade, tlme-on-job, aptitude, education, 
command, unit and base size, and age. All of these variables in 
combination yielded Rs which are only of modest size (.29 - .47). 
The largest and most consistent relationships are associated with 
the difficulty level of the work assigned and the aptitude level 
of the worker. These relationships are not large; but they are 
significant and always In the same direction. The most satisfied 
workers tend to be those who have the lowest relative aptitude and 
who are assigned to the most difficult work. In these tables, the 
variable "Work Difficulty" represents a least-squares weighted com- 
bination of (a) the number of tasks performed, (b) the number of 
tasks performed (squared), (c) the Average Task Difficulty Per Unit 
Time, and (d) a complex variable which is the sum of the cross- 
products of time spent on each task and the average grade level of 
all personnel In^the career ladder currently performing that task. 

The long-term job satisfaction research program of the 
Occupational Research Division involves three phases. First , we 
recognize that job satisfaction is multi-dimensional. We are 
attempting to isolate and define all significant job-related factors 
which should be included in our job satisfaction studies. Second, 
we want to determine the impact of each factor on career decisions. 
Finally , we want to determine how jobs and job conditions can be 
modified so that workers will look favorably on the Air Force as a 
career choice. 
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TABLE 10. 


PREDICTION OF 


REPORTED 


"UTILIZATION OF 


TALENTS 


AND TRAINING" 








BY FIRST 


-TEBM AIBMEN 








Career 




Full a 


Work® No. 


Tasks 






Ladder 


N 


Model 


Difficulty Performed 


ATDPUT^ 


Avg AI 


291 


862 


.44 


.21 


.21 


-.03 


-.20 


473 


720 


.36 


.28 


.21 


.18 


-.03 


543 


470 


.40 


.22 


.21 


.07 


-.21 


551 


836 


.45 


.32 


.28 


.30 


-.11 


605 


814 


.43 


.27 


.23 


.20 


-.16 


631 


876 


.36 


.19 


.07 


.15 


-.20 


645 


1,568 


.34 


.10 


.05 


.06 


-.18 


647 


1,470 


.34 


.14 


.11 


.12 


-.14 


702 


2,452 


.35 


.22 


.20 


.09 


-.18 


571 


1,214 


.29 


.14 


.07 


.13 


-.15 


811 


2,644 


.32 


.22 


.15 


.21 


-.10 



^See text for definition 

^Average Task Difficulty Per Unit Time 



TABLE 11. PREDICTION OF REPORTED "JOB INTEREST" 
BY FIRST-TERM AIRMEN 



Career 




Full® 


Work® 


No. Tasks 






Ladder 


N 


Model 


Difficulty 


Performed 


ATDPUT^ 


Avg AI 


291 


862 


.38 


.18 


.16 


.00 


-.14 


473 


720 


.40 


.26 


.15 


.22 


-.03 


543 


470 


.34 


.16 


.15 


.07 


-.14 


551 


836 


.47 


.28 


.25 


.23 


-.02 


605 


814 


.41 


.28 


.21 


.22 


-.13 


631 


876 


.40 


.24 


.06 


.20 


-.16 


645 


1,568 


.30 


.11 


.10 


.04 


-.06 


647 


1,470 


.32 


.16 


.12 


.13 


-.03 


702 


2,452 


.32 


.18 


.15 


.09 


-.08 


571 . 


1,214 


.30 


.19 


.09 


.18 


-.02 


811 


2,644 


.38 


.29 


.19 


.27 


-.06 



®See text for definition 

''Average Task Difficulty Per Unit Time 



ERIC 



42 



with respect to the first goal, we have developed a group of 
330 attitude statements which we feel cover the satisfaction domain. 
These are being analyzed using a combination of cluster analysis, 
factor analysis, and regression analysis, with the goal of producing 
a minimum set of attitude measures which cover all liiq)ortant dimen- 
sions. Phase II will Involve determining the Impact of each atti- 
tude on the reenllstment decisions of workers In each career ladder. 
This turns out to be a very complex problem. In the civilian 
sector, one can simply administer a job attitude questionnaire to 
a sample of workers and, at a later point In time, relate the 
score values to a criterion of "1 If still on the job; 0 otherwise.** 
In the military setting jobs are constantly being modified, and 
individuals are frequently moved from one location to another. We 
have no easy way of continuously tracking each Individual and 
measuring changes In job content, job conditions, and job attitudes 
over time. Furthermore, we don't know when each Individual 
finalizes his decision to reenllst or get out of service. 

We have developed a method, based on cross-sectional data, 
for Inferring the probable Impact of a particular job attitude on 
the reenllstment decisions of personnel In a particular career 
ladder. The model Is made possible by the fact that all enlist- 
ments In the Air Force have been for a A8-month period. It Involves 
construction of a regression curve which predicts the attitude of 
Individuals still on board at each month of military service. In 
order to afford some protection against the problems of Interpret- 
ing cross-sectional data In a longitudinal manner, aptitude Is held 
constant, and the regression line Is drawn for those at the mean 
aptitude level. 

To be more precise, assume that we are predicting the job 
Interest level for a particular group. The predictors included in 
the equation would be as follows; 

X]^ - Months of Service (continuous) 

=» 1 if Xj^ « 0-48; 0 otherwise 

X3 - 1 if Xi > 48; 0 otherwise 

X, ■ Xj^ if X2 = 1; 0 otherwise (or Xj^X2) 




Xg « Xi if X3 - 1; 0 otherwise (or X1X3) 
X- * Average Aptitude Index on the AQE 
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Note that the regression weights associated X2, X4, and X5 
will have an Impact only with respect to Individuals In their 
first enlistment. The weights associated with X^i Xg and X^ 
will Impact only upon Individuals who have elected to reenlist In 
the Air Force, and who are now beyond the 48th month of service. 

The upper left curve In Figure 8 presents a regression curve 
for a subsample of nearly 8,000 cases drawn at random from 130 
Air Force career ladders. Interpreting this curve In a longltu 
ditial manner, It appears that Air Force personnel have a slight 
decline In job Interest during their first enlistment. The jump 
In the curve between the 48th and 49th month of service Is hypo- 
thesised to be a function of reslduallzatlon. That Is, those who 
found tb.lr Jobs dull tended to get out In greater numbers than those 
who found their jobs Interesting. One might assume that the jump 
in the curve simply reflects a change In attitude by Individuals 
after they decided to reenlist; but this assumption Is weakened 
by thr observation that little or no jump occurs In the regression 
rart ^ cor many ladders. Regression curves for other ladd»-.:s are 
;V;>owa In Figures 9, 10, and 11. Where little difference l£ acl 
between the level of the regression curve at the 48th and 49 w . 
month. It Is assumed that efforts to make jobs more Interesting 
may have very little Impact on retention. Such Is the case for 
the 687X0, Programmer, where Individuals who left the service 
evidently were finding their jobs as Interesting as those who 
reenllsted. Perhaps some other factor, such as "pay In service 
compared with expected pay In comparable civilian jobs" would 
demonstrate a larger "Impact gap." If so, the service might better 
Improve retention by special pay benefits, or by educating workers 
concerning the reality of pay differentials. 

Hopefully, we will eventually come up with more direct measures 
of the Impact of job attitudes on retention. In the meantime, 
regression analyses, such as those described above, will provide 
clues as to what factors may Influence career decisions in each 
ladder. 

The third phase of the job satisfaction program, which is the 
most exciting, will be an attempt to find out what changes in jobs 
and job conditions will produce positive changes in those attitudes 
which Influence reenlistment decisions. Here the military services 
are in an ideal position to provide answers. Since jobs and job 
conditions are frequently changing, we can conduct Time 1 - Time 2 
studies in which we simply relate changes in jobs and job conditions 
to changes in expressed attitudes. 

Of course we recognize u. (Ranging jobs and job conditions 
is not the only approach to * * v^nci g job satisfaction. For example. 
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all vould agree that its better to marry a mate who id compatable 
than to marry one wlio is not, and try to change Kim or her. In a 
like manner, proper selection and classification actions can con- 
tribute toward future Job satisfaction. This is not a neglected 
area of research, but I simply don't have time to discuss It here 
today. Nor do I have time to discuss the various theories of job 
satisfaction and relate them to our research. An excellent paper 
on the implications of theories for Air Force job satisfaction 
research is currently in press (Tuttle & Hazel, 1973). 

As I mentioned earlier, factors related to job content, 
assignment preference, and worker-supervisor Interactions are 
among those frequently cited by personnel for their decisions to 
leave the Air Force. We are currently conducting an extensive study 
of the preferences of individuals for assignment locations. I can 
report that most individuals express very strong positive and negative 
valences for particular assignment locations. If the services cannot 
assign individuals to their most preferred location, perhaps they 
can at least avoid assigning them to locations for ^ich they have 
a strong negative valence. 

The matter of supervisor'-worker relationships is multi-faceted, 
and we will study each facet separately. One matter which should 
be of concern to all services is that of supervisory incompetency. 
We have evidence that occupational structures and personnel assign- 
ment practices can create situations where first-line supervisors 
have had no direct experience on critical tasks being performed by 
their subordinates. There is a particular danger of this occurring 
in complex career ladders involving numerous job types or having 
varied equipments to maintain. 

V. SUMMARY AND CLOSING REMARKS 

I have described the occupational survey techniques developed 
by the Air Force's Occupational Research Division. These tech- 
niques make extensive txse of job Inventories to collect information 
directly from workers in the field. I have described the CODAP 
analysis system, and given a few examples of how it can be applied 
to job survey data. I have presented a few examples of how job 
survey data can be coaibined with task and job difficulty informa- 
tion to obtain answers to questions posed by managers of the personnel 
system. 

I wish I had time to describe other research areas being pursued 
by the Occupational Research Division — including performance evalua- 
tion at the task level, definition of career progression routes; 
advanced assignment systems; job engineering; and so on. But I 
guess I must come to a close. 
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My understanding Is that you asked me to come to this symposium 
to share our experiences with you* I would like to close by repeating 
a set of recommendations, based upon experience, which I would make 
to any organization planning a large-scale job analysis system based 
on administration of task*- level job Inventories: 

!• Use full-*tlme Inventory writers to develop task lists and 
background questions • 

2. Write specific task statements, rather than bread task 
statements. 

3. Include any background Items which might answer questions 
asked by managers of the personnel system. 

4. Collect worker name and Identification Information. 

5. Administer Inventories to large samples. 

6. Collect data on optical scanning sheets. 

7. Use a "relative time spent" scale as the primary rating 
factor, and convert ratings Into percent time estimates. 

8. By all means, obtain and use the CODAP system If at all 
possible. You can modify It to your needs, but It will be expen- 
sive and time consuming to build your own analysis system. 

9. Establish an occupational research group to develop 
applications of occupational data In your military service or 
organization. 

Thank you for the opportunity to participate in this symposium. 
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APPENDIX 

The following pages present a brief description of selected 
CODAP programs, along with example printouts produced by the CODAP 
system, as follows: 

1. The first four pages of a seven"*page Consolidated 
Job Description. 

2* An Abbreviated Job Difference Description. 

3. A sample Individual Job Description. 
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DESCRIPTION OF SELECTED CODAP PROGRAMS 



CODAP: COMPREHENSIVE OCCUPATIONAL DATA ANALYSIS PROGRAMS 

cdbAP is a computerized occupational data analysis system which inputs and performs calculations 
upon raw c .ta from job inventories. It is designed to furnish users with a wide variety of reports that 
facilitate the identincation of individual and group job characteristics and the detection of between-job 
similarities and differences. 

INPSTD: RAW DATA EDITING AND INPUT 

This program reads task titles, task responses, and background data from tape or card input. It edits 
the data, converts the raw task responses to percentages, constructs data vectors for each case, reorganizes 
the data to a standard hbtory data format, and writes the formatted data on the output tape for use in 
subsequent programs. INPSTD will accept a maximum of: 928 background and/or computed variables, 
1000 task variables, 26 duty variables, 20,000 cases, and 927 non-zero task responses per case. 

JOBDEC: CALCULATING COMPOSITE JOB DESCRIPTIONS 

This program calculates and prints composite job descriptions for groups formed during the 
hierarchical grouping process (JOBGRP) or for special groups whose membership is defined in terms of 
some combination of background or computed variables (JOBSPC). Both duty and task job descriptions 
may be reported in high to low sequence of either "average percent time spent by all members** or **peicent 
of members performing.*' (A duty is a functional area comprising a number of tasks and, hence, is a 
summaiy report). 

A job description produced through JOBDEC provides the following information: duty/task number, 
duty/task title, percent of members performing each duty/task, average percent time spent by members 
performing, average percent time spent by all members, and cumulative average percent time spent by all 
members. 

GRPSUM: SUMMARIZING JOB DESCRIPTIONS 

This program calculates and prints a report of either the percent of members performing each task in 
the job inventory or the average percent time spent on each task by all members for any number of groups 
whose composite job descriptions were computed by JOBDEC or JOBSPC. The summarized data is printed 
in task number order and^ group descriptions are ordered according to the sequence of the input request 
cards. GRPSUM does not print task titles, as does PRIJOB, nor does it have PRIJOB's facility to specify 
criteria for excluding tasks from the sunmiary report. 

GPSUM: SUMMARIZING JOB DESCRIPTIONS (FORMAT 2) 

This program is a version of GRPSUM containing certain desirable features adapted from PRIJOB. 
Like GRPSUM, GPSUM2 calculates and prints a report in duty/task order of either the percent of members 
performing each task in the job inventoiy or the average percent of time spent on each Usk by all members 
of groups whose composite job descriptions were computed by JOBDEC or JOBSPC. Whereas, GRPSUM 
prints no task statements and canies all percentages to three decimal places, GPSUM2 does print task 
statements and rounds off all percentages to whole numbers. 

PRUOB: CALCULATING PRIMARY JOB IDENTIFIERS 

This program calculates and prints a report of those tasks which are determined to be "primary 
identifiers** of job types. Primary identifiers may be defined as the top x-number of tasks in a group job 
description in terms of percent of members performing or average percent time spent by all members. 
Primary identifiers may also be defined as those tasks perfomwd by a specified minimum percentage he 
group members or whidi exceed a specified average percent time spent value. This program allows a nu . r 
of groups to be aligned in a single report for comparative purposes. 
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JOBIND AND INDIOB: CALCULATING INDIVIDUAL JOB DESCRIPTIONS 

The JOBIND program calculates and prints job descriptions for individual cases. Both duty and task 
descriptions may be reported. Output will be task statements sequenced from high to low percent time 
spent, together with their percent time spent values and cumulative percentages. Selected background 
information may also be printed at the top of each description. JOBIND has six options for selecting cases: 
( 1) by last two digits of service number; (2) by every Nth case beginning with a specified KPATH sequence 
number; (3) by meeting specified requirements on one to nine background variables; (4) by membership in 
a specified job type group; (5) by case control number; (6) randomly, using a random number generator. 
The selected descriptions may be sorted in KPATH order, in random order, by background variable, or in 
case control number order. The six options for selecting cases may also be used to create new categorical 
membership variables which are inserted in the variable dictionary as computed variables. 

The INDJOB program also calculates and prints individual job descriptions, but prints only task 
identification numbers and percent time spent values for duties or tasks in duty/task sequence. A single 
report prints information for all cases in a columnar format. There are only three options for selecting 
cases: (1) by case control number; (2) by meeting specified requirements on one to nine background 
variables; (3) by membership in a specified job type group. INDJOB has no sort options. 

GRPDIP: DIFFERENCE COMPARISON BETWEEN JOB DESCRIPTIONS 

This program calculates and reports the difference between two job descriptions in terrhs of 
percentage of members performing each task and/or average percent time spent. Deference values are 
presented in ascending or descending order on either value (from largest negative to largest positive 
difference or vice versa) or in task number order. 

AUTOJT: AUTOMATED JOB TYPE SELECTION PROGRAM 

This program calculates, evaluates, and reports between*group differences for specified pairs of groups 
whose job descrq)tions were computed by JOBDEC or JOBSPC. Six comparison options are used: (I) 
differences in average percent time spent on each task; (2) differences in average percent time spent on each 
duty; (3) differences in percentage of members performing each task; (4) differences in number of tasks 
accounting for a major portion of average percent time spent; (5) differences in number of duties 
accounting for a majw portion of average percent time spent; (6) differences in average number of tasks 
performed. As many as 850 pairs of groups can be compared in one run of AUTOJT. 

VaRSUM: SUMMARIZING BACKGROUND AND COMPUTED VARIABLES 

This program computes and reports frequency distributions within specified intervals, makes total 
frequency counts, and calculates means and standard deviations on selected background and computed 
variables for any group of individuals whose job description has been generated by JOBDEC or JOBSPC. 
VARSUM can process data on as many as 20,000 cases. 



DIST2X: COMPUTING A TWO-WAY DISTRIBUTION 

This program distributes a group of individuals on two variables (a row variable and a column 
variable). The group distributed may be the total sample or the cases coded "T* or "0" on a categorical 
membership variable. The row and column variables may be alpha or numeric, and intervals of unequal 
width may be defined for any variable. The data presented may be frequencies within cells or frequencies 
and percentages within cells. The percentages are computed based on individual row frequency totals, or on 
individual cohimn frequency totals, or on the total N for all rows and columns. Any or all three sets of 
percentages may be displayed in the two-way table. A "total'* category for rows and a "total" category for 
columns is automatically included for percentages and/or frequencies. The total number of cases counted 
and the total number of valid cases are given for each row and column and for total rows and total columns. 
Valid cases include dSl distributed numeric data. Only valid cases are used in the computation of means and 
standard deviations. Optionally, an "other'' category may be included for rows and columns. Inclusion of 
the "other" category causes all numeric data to be counted as valid. Another option provides fc- the 
con4)utation of means and . standard deviations for individual rows and total rows and/or individual cc mris 
and total columns. Only cases defined as valid are used in computing means and standard dcviat i. A 

O summary of selected information appears at the end of the report. Any number of reports may be 

J generated in one program execution. 
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AVALUE: CALCULATING AVERAGE VALUE 

For each task in a job inventory, this program calculates and prints the mean and standard deviation 
of a selected background or cmiputed variable, using all valid responses of individuals in a specified group 
who perform the task. Task titles and the number of valid respondents on each task are alao reported. 
Average values are optionally sequenced from low-to-high average value, high-to>low average value, or in 
task numbu^r order. Tasks with fewer than x-number of valid respondents may be removed from the main 
report to a supplementary report. AVALUE can be calculated for any group of individuals who can be 
identified on some range of a background or computed variable or by the interaection of up to nine 
variables. 

TSKNDX: CALCULATING AVERAGE TASK RATINGS 

This program has the same options as AVALUE, except that it is used with task ratings rather than 
with background or computed variables. TSKNDX can also provide the following additional task 
information not available in AVALUE: (1) percent of members performing; (2) average percent time spent 
by members performing; (3) average percent time spent by all group members; (4) cumulative sum of 
average percent time spent by all group members. TSKNDX is primarily used to compute average task 
difTiculty ratings. 

RXXNDX: COMPUTING INTERRATER RELIABILTTV 

This program computes and reports for a group of raters the average interrater teliability coefficient 
of a single rater and the stepped up reliability coefficient for the total group of raters. The program is most 
often used in conjunction with sets of task difficulty ratinp made by a large numberof supervisory 
personnel. 

TSKDST: DISTRIBUTING MEANS AND STANDARD DEVIATIONS OF TASK RATINGS 

This program computes and reports the distribution, mean, and standard deviation of thiemean task 
ratings and the standard deviations of the ratings for each task. Tiidnprto T9CDST is die punched 
output furnished by TSKNDX. 

ASF ACT: REPORTING SECONDARY FACTOR DATA 

If the tasks in a job inventory are rated on a second scale such as '^amount of training required/* the 
data is handled throu^ the ASFACT program. The ASFACT program reports the following information on 
each task for any group of individuals vAiox job description has been computed by JOBDEC or JOBSPC: 
(1) frequency distribution of members responding on the secondary factor (0 to 9 scale); (2) total number 
of respondents; (3) number of respondenU with values outside the specified range; (4) arithmetic mean and 
standard deviation of acceptable responses. 

VARGEN: COMPUHNG NEW VARIABLES 

This program calculates new computed variables by applying input data to the task values of each 
case. The task values may be time spent percentages or "do - don't do" values (1,0). The input data 
consists of a vector of weights* a scaling (standardizing) factor and a specified calculation formula (five 
options). A newdy created variable is given a variable identification number and is added into the computed 
variable portion of the case data records on the history data tape. 

PREGEN: GENERATING NEW PREDICTORS 

This program cieates new computed ox background variables for input to other CODAF programs, 
principally the correlation and regression program (CORREG) according to certain standardized options, as 
follows: the new variable can be the sum of two variables, the difference between two variable^ the 
product of two variables, or the ratio of two variables. Each option allows for adding a constant t so 
desired. Variables may also be created by setting certain values of a variable out of range and then, cuher 
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leaving these values unchanged during the generation of the new variable, or setting them equal to zero or 
to some constant. This option is most frequently used to enable the selection of a specific subsample of 
cases by creating a categorically coded membershq) variable representing in-range and out-of-range cases 
CM** or ^^0/* respectively). Variables created at an early point in the program run may be used to create 
additional variables at some later point. Up to 500 variables can be created in a single PREGEN run. 

GDRREG: CORRELATION AND REGRESSION PROGRAM 

This program package extracts up to 100 computed and background variables from a CODAP KPATH 
or history data Xgpe and computes correlation matrices and regression problems. 

The correlation program computes and prints the correlation matrix, number of valid and invalid 
cases in the sample, and means and standard deviations of variables. These same computations are also put 
on tape for future reference, and they also remain in core if they are to be fed into the regression program 
immediately. 

The regression program has two options. One option computes regression equations for various 
combinations of full and restricted models and evaluates the difference between full and restricted models 
with an F-test. The value and the probability of the computed F-ratio are reported. 

The second option is designed as an aid to building an appropriate regression model. In this option, a 
series of regression problems is computed iteratively. Beginning with the best combination of three 
predictors at iteration 1, the best remaining combination of three predictors is added to the model at each 
subsequent iteration. The ''best remaining combination'* is that whkh adds the most to the value of 
when used in conjunction with all combinations of predictors selected at previous iterations. Predictors may 
be used more than once during tlie iterative process.. Iterations are continued until the increase in the value 
of over the previous iteration is less than some amount specified by the requester. Variable ID's, R^ 
values, and the error sum of squares are reported for each iteration. The standard and raw score weights for 
each variable as they exist upon completion of the final iteration are reported, as well as the regression 
constant. 

Either one or both regression options may be requested at the same time. 

OVRLAP: RELATING RFSPONSES TO EACH OTHbR 

This program generates an overlap or similarity matrix of all possible paired comparisons between 
individual cases. Similarity is expressed as a percentage of common tasks performed (TSKOVL) or as the 
total overiapping percentage of time spent on tasks (TIMOVL). Overlap in terms of percent time spent is 
the preferred option in most studies. OVRLAP can handle up to 2,000 cases and 1 ,000 tasks. 

GROUP: CLUSTERING INDIVIDUALS ANn GROUPS OF INDIVIDUALS 

This program uses the similarity matrix computed in the overlap (OVRLAP) program to form clusters 
of cases. The grouping technique, called ''collapsing ihe matrix'* or "hierarchical grouping,'* involves 
repeated searching for those individuals or partiaUy fonned clusters which have the highest (or lowest) 
remaining similarity, depending on whether a '"maximizing" or ''minimizing'' process was requested. The 
"maximizing** option is always used for job survey data. 

Each new clustering or "collapse** is called a "stage** and the vectors of similarity values for the 
clusters being merged are combined according to a specified mathematical algorithm to form an integrated 
cluster. Several formulas for combining groups are available. The collapsing process continues until a single 
group has fonned which contains aO cases in a study. 

KPAm ANDPRKFTH: ORDERING A HISTORY DATA TAPE AND 
PRINTING A HISTORY DATA REPORT 

After OVRLAP and GROUP have been completed, the KPATH program assigns sequence numbers to 
individual cases in such a w^ that each pair of individuals or groups which are merged during the gro' 
process will have a contiguous block of KPATH sequence numbers. 
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PRKPTH enables the user to select variables and printout fomiats to produce a report of the case data 
values for the selected background and computed variables. The data to be printed is obtained from a 
history data tape which may be in case ID or KPATH order. The data is not sorted and therefore the output r 
will be in case ID or KPATH order. 

DUVARS: COMPUTING A DUTY VARIABLE FOR EACH CASE 

This program uses task data to compute duty values for each case and displays the duty values in the 
form of a PRKPTH. Three duty variable options are available: ( 1) percent time spent in each duty by each 
case; (2) number of tasks performed in each duty by each case; (3) percent of tasks performed within each 
duty by each case as a function of his total number of tasks performed. This program is used principaDy as 
an aid in selecting meanii|gfuUy different job-type groups. 

GRMBRS: REPORTING GROUP MEMBERSmP 

This program produces an information report that identifies the two groups combining at each stage 
of the hierarchical grouping process. The information includes the stage number, the number of members in 
the combined group, the number of members in each of the combining groups, range of KPATH sequence 
numbers for the combining groups, the average percentage of overiap between the members of the 
combining groups, and the average percentage of overlap within the combined group. 

Dl AGRM: GRAPHICAL PRESENTATION OF HIERARCHICAL GROUPING ACTIONS 

This program uses data from the GRMBRS program to generate a treelike diagram that visually 
displays the order in which groups merged during the hierarchical grouping process. Each group is 
represented by a rectangular block of data containing the same information found in GRMBRS. Rows and 
cclumns of asterisks show the branches leading from a group to its subgroups. Control cards can be used to 
limit the number and type of groups displayed by DIAGRM. 

MBROVL: OVERLAPPING INDIVIDUAL JOB DESCRIPTIONS 
WITH A C0MPOSI1E JOB DESCRIPTION 

This program computes the overlap of individual job descriptions with the composite job description 
for the group and osports the individual overlap values, their mean and standard deviation, and an array of 
selected background data for each case. The format of the report is similar to PRKPTH, except that the 
cases are sequenced from hi^est to lowest overlap with the composite job description and only cases that 
are members of the selected group are included. Reports on a number of groups can be handled in one 
program execution. This program is useful in studying the homogeneity of memberdiip in a group whose 
composite job description was computed by JOBDEC or JOBSPC. 

MTXPRI: PRINTING AN OVERLAPPED MATRIX 

This program calls for the subroutine OVRLAP to overlap all possible pairings of a set of composite 
(group) job descriptions and then uses the program MTXPRT to print the between-group overlap values in 
mairix fom. Overlap may be computed in terms of average percent time spent on tasks or in terms of 
number of tasks performed in common. The maximum number of groups that can be input to MTXPRI is 



JD2HDT: ADDING JOB DESCRIPTIONS TO HISTORY DATA TAPE 

This program adds average job descriptions for groups to a history data tape as additional cases. Eadi 
of the new compoate cases is given the next sequential case control number. 
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PRDICr: PRINTING DICTIONARY OF VARIABLE TrrtES 

This program prints a report containing the identification codes and descriptive titles of all 
background and computed variables peculiar to a particular study. The identiflcation codes are used in 
calling for data to be reported by the PRKPTH and VARSUM programs, or to be acted upon by the 
VARGEN, PREGEN, PROGEN, JOBIND, AVALUE, or TSKNDX programs. 

** JOBINV: PRINTING OF DUTY AND TASK TITLES 

This program prints a listing of the duty and task titles included in a job inventory. The titles are 
listed in task number sequence within each duty, and the format calls for two columns per page. 

BCDEXT: REORDERING AND EXTRACTING REPORTS FROM THE BCD REPORT TAPE 

All reports generated by CODAP can optionally be placed on a BCD output tape for future recall. 
The BCDEXT program extracts reports selected by conuol cards from the BCD fOe, prints them out in the 
same order as the input requests with continuous page numbering^ and furnishes a table of contents with 
page number references. 

PROGEN: PROGRAM GENERATION PROGRAM 

This program permits the CODAP expert to add, extract, and manipulate data in the CODAP ^stem 
in ways not encompassed by the standard CODAP programs and do so with a minimum amount of 
additional programming and without requiring thorough indoctrination in the CODAP system. PROGEN 
uses a combination of FORTRAN statements and shorthand operation codes to generate a special purpose 
program to read the KPATH or history data tape, either of which contains the entire data fQe for each caae, 
and perform operations upon it New variables can thereby be created and added to the CODAP variable 
dictionary. This program also has the facility to reconvert percent time task values for each case to the 
original raw response form, perform operatk>ns on the raw responses, and convert them back to 
percentages. 
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TASK JOB DESCRIPTION FOR JOURNEYMAN MEDICAL LABORATORY SPECIALISTS (N-3B4) 

CUMULATIVE SUmV AVERAGE PERcb^TIIM^^ . . 

AVERAGE PERCENT TIME SPENT BY ALL MEMBERS • 
AVERAGB PERCENT TIME SPENT BY MEMBERS PERFO 
PERCENT OF MEMBERS PERFORMING . • • 

TASK TITLE 

Colluct Blood Specimens Diiectly from Patlenti 
Perform Blood Count 

Perform Hematology Pioceduies for Diffmntial Cell Counts 
Perform Hematology Procedures for Hematocrit Tests 
Lxaniine Urine Specimens MicroKopicaOy 
Prepare Blood Smears 
Prepare and Process Specimens 
Perform Urinalyses for Glucose Tests 
Perform Urinalyses for Speciflc Gravity Tests 
Perform Urinalyses for Albumin Tests 
Ckan Area and Equipment Aseptically 
l:.xamine Urine Specimens Macroscopically 
Separate Serum from Blood 
Prepare Reagents 

Identify Morphological Variations of Blood Cells 
Operate Spcctro-Photometer 

Perform Hematology Procedures for Erythrocyte Sedimentation Rate 
Perform Serological Procedures for Cardiolipin Miaoflocculation 
Fxamine Specimens Miaoscopically 
Identify and Classify Pathogenic Bacteria 
Prepare Culture Media 
Prepare Solutions and Standards 
Perform Biochemical Procedures for Livei Function Tests 
Perform biochemical Procedures for NPN and BUN Tests 
Slain Hacteriological Smears 
Crossnutch Blood 

Text Blood for ABO Grouping and ABOSubgrouping 
Identify Immature Blood Cells 
l Aainine Specimens Microscopically 
Perform Antibiotic Sentitivity Test 
Prepare Specimens for Shipment 
Log Incoming or Outgoing Specimens 
Type Blood of Donors and Recipients 
Cenlrifuge and Separate Scrum from Clot 
Perform Biochemical Procedures for Total Protein and A/G Ratio 
Test Blood for RHO or DU Factors 
Perform Direct and Indirect Coombs Tests 
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CUMULATIVE SUM OP AVERAGE PERCENT TIME SKNT iY ALL MEMBERS 

AVERAGE PERCENT TIME SPENT iV ALL MEMiERS 

AVERAGE PERCENT TIME SPENT iV MEMiERS PERFORMING • 

PERCENT OP MEMBERS PERFORMING 
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I 


< 


Diaw Blood for Transfusions 


64.47 




A Sll 


SA 111 


K 


13 


Perform Serological Procedures for Heterophlle Presumptive and DifferentM 












Antibody Test 


63.4S 


0.90 


0.57 


56.94 


J 


It 


Perform Hematology Procedures for Eosinophile Count 


80.46 


0.71 


0.57 


57.51 


M 


2 


Operate Flame Photometer 


64.97 


OM 


0.57 


58.08 


G 


8 


Perform Sperm Counts 


79.44 


0.71 


0.57 


58.65 


J 


29 


Perform Hematology Procedures for Sidcle Cell Preparations r 


82.74 


0.68 


0.56 


59.21 


M 


14 


Perform Biochemical Procedures for Carbon Dioxide Determinations 


67.26 


0.83 


0.56 


59.77 


£ 


11 


Receive Incoming Supplies 


SSSB 


0.96 


0.53 


60.31 


L 


IS 


Store Blood According to Grouping and Factor 


S9.90 


0^9 


0.53 


60.84 


P 


20 


Collect Pus Specimens Directly from Patientt 


65.99 


0^0 


0.53 


61.37 


N 


20 


Perform Urinalyses for Urobilinogin Tests 


75.89 


0.66 


0.50 


61.87 


K 


14 


Perform Serologtcal Procedures for Latex Fixation Test 


59.64 


0J4 


0.50 


62.37 


K 


6 


Perform Serological Procedures for '"C** Reactive Protein Testa 


61.42 


0^0 


0.49 


62.86 


II 


4 


Perform KOH Preparation for Dermatophyte 


68.02 


0.72 


0.49 


63.35 
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CUMULATIVL SUM OF AVERAGE PERCENT TIME SPENT BY ALL MEMBERS . . , 

AVhRAGE PERCENT TIME SPENT BY ALL MEMBERS 

AVERAGE PERCENT TIME SPENT BY MEMBERS PERFORMING 

PFiRCENT OF MEMBERS PERFORMING • • • • • 



IMSK 


TASK TITLE 


• 
• 


• 
• 


• 
• 


• 
• 


A 


\u 


IX'vclop unJ Improve Woii Methods and Procedures 


53.55 


0.91 


0.49 


63,84 


1. 


4 


l)isp«>su of Blood After Time Limit 


62.18 


0.77 


0.48 


64.32 


M 


K 


IVrl'orni Biochemical Procedures for Blood Alcohol Tests 


66.75 


0.71 


0.48 


64.79 


M 


20 


Perform Biochemical Procedures for Creatinine Tests 


61.42 


0.76 


0.47 


65.26 


1 


6 


Maintain Donor Tiles 


58.63 


0.79 


0.47 


65.73 


N 


10 


Perform l/rinalyses for KETONE Studies 


55,33 


0.84 


0.46 


66.19 


11 


2 


1 \:imine Specimens Microscopically 


60.15 


0.77 


0.46 


66.65 


.1 


12 


Perform Heinatolocy Procedures for Clot Retru^tion Test 


73.35 


0.63 


0.46 


67 J 1 


A 


5 


Assure the AvailabiUty of Hquipmcnt and Supplies 


42.64 


1.06 


0.45 


67.57 


A 


26 


Requisition Supplies and Equipment 


44.67 


1.01 


0.45 


68.02 


1- 


12 


Requisition Supplies 


44.42 


1.00 


0.45 


68.46 


N 


16 


Perform Urinalyses for Total Protein 


63.45 


0.70 


0.44 


68.90 


M 


31 


Perform Biochemical Procedures for Enzyme Analyses 


46.70 


0.95 


0.44 


69.35 


M 


^2 


Utilize Methods for Titrimetric Procedure 


55.33 


0.80 


0.44 


69.79 


M 


1 J 


Perform Biochemical Procedures fArPartuiltvtflrat^t TnlMttn^i tm** 


44.67 


0.98 


0.44 


70,23 


II 




Prepare Culture Media 


57A7 


0.76 


0.44 


70,67 


11 


1 


Cultivate Mycology Specimens for Primary Isolation 


56.09 


0.77 


0.43 


71,10 


D 


6 


Give On~Thc-Job Instruction in Medical 1 atmntntv ArtivftiM 


40.10 


1.04 


0.42 


71,51 


N 


7 


Perform llrinalv^ec for Rpnr^.lnn^c PmftMin Xm*» 


68.78 


0.60 


0.41 


71 93 


1 


5 


Stain Paiasitoloffical Smears 


53.81 


0.77 


0.41 


72,34 


I* 


22 


Collect Skin Soccimens Rirectlv frnm Patient* 


58.12 


0.71 


0.41 


72,75 


K 


8 


Perform Serological Procedures for Cnld A Mluti nations 


57.U 


0.72 


0.41 


73.16 


N 


4 


Perform Pregnancy Tests 


48.48 


0.84 


0.41 


73,57 


C 


6 


ICvaluatc the Accuracy of Routine Reports 


39.09 


1.04 


0.41 


73,98 


K 


5 


Perform Scroloeical Procedures for AntUtr^ntnlvdn Tit«n 


48.48 


0.82 


0.40 


74,37 


1. 


13 


Record Information on Blood Record Card 


53.05 


0.74 


0.39 


74.77 




7 


Maintain Files of Blood Banking Forms 


53.30 


0.74 


0.39 


75.16 


1 


9 


Perform Preventive Maintenance on Laboratory Equipment 


47.72 


Oj82 


0.39 


75,55 


1 


24 


Collect Sputum Soccimens DirectIv frnm Paii^ntc 


52.28 


0.72 


0.38 


75,93 


1. 




Attach Serial Numbers to Units 


48.22 


0.78 


0.38 


76,30 


M 


1 


Calibrate Instruments 


52.03 


0.72 


0.37 


76,68 


1. 


U 


Screen and Schedule Donors 


50^1 


0.72 


0.36 


77,04 


N 


5 


Perform Urinalyses for Addis Counts 


63.96 


0.56 


0.36 


77,40 




1 


Determine Equipment Repairs or Replacements Needed 


47.21 


0.76 


0.36 


77,76 


o 


9 


Prepare Specimens for Shipment 


39.85 


0^9 


0.36 


78.12 


L 


II 


Prepare Blood for Shipment 


46.70 


0.72 


0.34 


78,46 


M 


40 


Utilize Methods for Gasometric Procedure 


41.37 


0,81 


0.34 


78.79 


K 




Perform Serological Procedures for Febrile Aggluttiutions 


45.69 


0.72 


0.33 


79.12 


A 


21 


Plan Reports for the Section 


32.99 


0.99 


0.33 


79.45 


i: 


4 


Maintain and Revise Stock Levels 


35.53 


0.92 


0.33 


79.77 


A 


20 


Plan Record Keeping for the Section 


30.71 


1.06 


0.33 


80.10 
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CUMULATIVE SUM OF AVERAGE PERCENT TIME SPENT BY ALL MI MBERS • • • 

AVERAGE PERCENT TIME SPENT BY ALL MEMBERS 

AVERAGE PERCENT TIME SPENT BY MEMBERS PERFORMING 

ri'RCI NT OE MEMBERS PERFORMING 



i>TSK 


TASK TITLE 




• 


• 


• 


J 


19 


Perform Hematology Procedures lor Eryiluocytc l-ragitity Tests 


59 14 


0.55 


0.32 


K0.42 


M 


30 


Perform Biochemical Procedures for Scrum Prog Test for Pregnancy 


40.61 


0.78 


0..^2 


H0.74 


1' 


6 


Perform Bacteriotogicul or Chemical Examinations of Water 


41.37 


0.74 


0 31 


HI. 05 


il 


6 


Stain Mycology Specimens 


48.23 


0.62 


0.30 


81 34 


N 


17 


Perform Urinalyses tor Urinary Calcium 


S4.57 


0.54 


0.30 


81.64 


N 


14 


Perform Urinalyses for Porphyrins Tests 


54.57 


0.54 


0.30 


81.94 


if 


3 


Maintain Slock Cultures 


35.79 


0.82 


0.29 


82.23 


C 


7 


Evaluate the Adequacy of Routine Reports 


29.44 


0.98 


0.29 


82.52 


() 


IS 


Submit Tissue Specimens to AI-IP or llistupathotogy Centers 


32.99 


0.87 


0.29 


82.81 


A 


7 


(Coordinate Work Activities with Other Sections 


36.55 


0.77 


0.28 


83.09 


A 


14 


llstablish Procedures for Special J ests 


36.29 


0.74 


0.27 


83.36 


II 


2 


Direct Subordinates in Maintaining Performance Standards 


30.96 


0.87 


0.27 


83.63 


i: 


to 


Procure and Store Biological Items 


35.53 


0.75 


0.27 


83.89 


ti 


3 


Identify and Classify l- ingi 


36.04 


0.73 


0.26 


84 16 


o 


2 


Assist with Autopsy 

Perform Bacteriological or Chemical Examinations of Food Products 


39.34 


0.66 


0.26 


H4A2 


i>' 


4 


4C.d6 


063 


0.26 


84.67 


I. 




Perform First Aid for Shock 


51.02 


0.49 


0.25 


84 92 


K 


2 


Prepare Antigens 


32.49 


0.77 


0.25 


85.17 


I- 


IS 


Prepare Specimens for Traii'ung or Reference 


36.29 


0.67 


0.24 


85 .42 


N 


13 


Perform Urinalyses for Pheiiylpyruvic Acid Test 


46.9S 


0.52 


0.24 


H5.66 


1- 


1 


Supervise the Maintenance of Laboratory Supplies 


23.60 


1.02 


0.24 


85 90 


H 


s 


Direct the Maintenance and Utflization of Equipment, Supplies and Work Space 


27.92 


0.86 


0.24 


86.14 


A 


3 


Assign Specific Work to Individuals 


30 96 


0.78 


0.24 


86.38 


<• 


IK 


Resolve Technical Problems of Subordinates 


28.43 


0.83 


0.24 


86.62 


M 


19 


Perform Biochemical Procedures for Creatinine Clearance Tests 


32 23 


0.73 


0.23 


86.85 


|> 


X 


Indoctrinate Newly Assigned Personnel 


35.28 


0.67 


0.23 


87.09 


<* 


14 


Investigate Possible Sources of Staphylococcus Outbreaks 


28 43 


0.82 


0.23 


87.32 


A 


25 


Plan Work How 


25 13 


0.92 


0.23 


87.56 


(• 


9 


I valuate Work Performance of Subordinates 


23.35 


0.95 


0.22 


87.78 


M 


15 


Perforin Biochemical Pnvcdures for Carbon Monoxide Determinations 


39.09 


0.57 


0.22 


88.00 


1 


K 


Prepare (Culture Media 


29.70 


0.73 


0.22 


88.22 


A 


II 


Develop or Revise the Organization of the Section 


26.40 


0.81 


0.21 


88.43 


U 


4 


Direct Subordinates in the Observance of Safety Practices 


27.66 


0.77 


0.21 


88.64 


J 


9 


Perform Hematology Procedures for BleedingTime, Ivy Method 


29.44 


0.71 


0.21 


88.85 


l> 


18 


Show flow to Locate and interpret Technical Information 


25.89 


0.78 


0.20 


89.06 


A 


18 


Plan and Schedule Work Assignments 


24.11 


0.83 


0.20 


89.26 


M 


29 


Perform Biochemical Procedures for Salicylate Level 


2.49 


0.61 


0.20 


89.46 


C 


16 


Recommend Special Corrective Action for Recurring Problems 


2( .65 


0.72 


0.19 


89.65 


C 


K 


Evaluate the Maintenance and Use of Equipment, Supplies and Work Space 


23^6 


0.80 


0 19 


89.84 


N 


18 


Perform Urinalyses for Urinary Chlorides 


35 03 


0.54 


0.19 


90.03 
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SAMPLE DIFFERENCE DESCRIPTION 

GROUP 1 =^ APPRENTICE DENTAL LABORATORY TECHNICIANS (N-30) 
GROUP 2 - JOURNEYMAN DENTAL LABORATORY TECHNICIANS (N«272) 







nil 1 1 Rl NCi IN PI RC TNT PbRI ORMING GROUP 


1 MINUS GROUP 2 










n R( 1 NT ri:RI ORMING.i;ROUP 1 . .... 












ri RC I NT PI:RI ORMING, GROUP 2 








1) ISK 


TASK TITLE 






• 


K 


It 


IVrioiin hcniiil A>>i<it.Mit 1 uiKik)ns 


12.87 


33.33 


20.47 




{ 


Mi\ ,int! t»ri |Mrc' Sliirn \V:iKr 


51.10 


70.00 


18.90 


11 


14 


1 I.inK i i>iiiptcU' l)oiitiirc> 


54.78 


73.33 


18.55 


M 




M.iini;iiii Hoitoul i;ink> 


52.57 


70.00 


17.43 


(; 


in 


Miv i»r l*rc|>jrc Uuplk-jtiii)! MjIctijIs 


20.96 


36.67 


15.71 


1 


} 


IVt p.iir MalriN for Di iittfrc Kcputr 


54.41 


70.00 


15^9 


M 


7 


Muinl:iin iKliyilrulinp 1 Miiipnicnt Ovens 


11.40 


26.67 


15.27 


II 


^0 


l';K-k Ally Mi Dciiturcv 


54.78 


70.00 


15.22 


II 


10 


( lire uiul Dclhisk Complete Dentures 


55.51 


70.00 


14.49 


tl 


2U 




55.51 


70.00 


1449 


1 


2h 


Trim :im1 W:i\-l)ip Relrjc tory ( jstsof Removable PjrtijI Dentures 


13.34 


26.67 


13.43 


II 


y 


Ar(k(ib(e('uve<i 


63.60 


76.67 


13.06 


M 


1 1 


MiriiinjU* W;i\ from Denture Molds 


56.99 


70.00 


13.01 




4 


IU';iil. liov. and Pour 1 inji Impression to Produec Stone Master Cast 


47.06 


60.00 


12.94 


II 




('<in\iruit Trial Hjsepljtes jnd Bite Rims 


49.26 


60.00 


10.74 


1 


21 


Si»;ik M;ister Casts 


29.78 


40.00 


10.22 


M 


13 


M;iiniain lianati Art aula lor iind Articulator Mounting Plates 


53.31 


63.33 


10.02 


l ASKS OMITTI 1) Will Ri: DIPf l,R| NCLS IN PI- HCLNT PERFORMING = 10.00 THROUGH -20.00 






K 


17 


Solder Units ol 1 i\ed Partial Dentures 


33.46 


13.33 


-20.12 


!{ 


4 


Siipt rvise Dental Laboratory Specialists (APSC 98250) 


20.22 


0.00 


•20.22 


M 


1<> 


Maintain Manu;il C:istin^ Machines 


33.82 


13.33 


•20.49 


K 


9 


1 ubricjie Stone Dies 


40.81 


20.00 


20.81 


\ 




Replace 1 ube IVeth or l aeingsv 


41.18 


20.00 


•21.18 


K 


15 


Tickle or Meat 1>eal Cold Inlays. Crowns, or Pontics 


37.87 


16.67 


•21.20 


K 


1 


Cast (U)ld Crown. Inlay, or Pontic Backing 


38.24 


16.67 


-21.57 


K 


1^ 


Test Occlusion and l it of Inlays. Crowns, or Fixed Partial Dentures 


38.60 


16.67 


•21.94 


1 


8 


Repair Metal Parts of Removable Partial Dentures 


25.37 


3.33 


•22.03 


li 


13 


Supervise the 1 abrieatton of Dental Prosthetic Appliances 


22.06 


0.00 


'22.06 


k 


1(» 


1 inish and Polish Gold Alloy Inlays. Crowns^ or Fixed Partial Dentures 


38.97 


16.67 


•22.30 


K 


iH 


Sprue. Invest, and litirn Out Cold Alloy Inlays, Crowns, or Pontics 


38.97 


16.67 


•22.30 


K 


13 


Crind in Porcelain or Acrylic I'acingsand Pontics 


39.71 


16.67 


•2304 


k 


> 


CtMKtrtiei and Articulate Casts 


43.38 


20.00 


•23.38 


K 


s 


1 ahricate Acrylic Resin Jacket Crowns 


32.72 


6.67 


•26.05 
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EXAMPLE INDIVIDUAL JOB DESCRIPTION 

CASE CTRL NUMBER > 1134 

NAME * WITHHELD 

GRADE «E.3(A1C» 

TOT MOS AFMS « 015 

NO' SUBORDINATES > NONE 

MAJOR COMMAND « AIR FORCE SYSTEMS COMMAND 

PRES WORK ASGNMT » CIVILIAN PAYROLL CLERK 

EDUCATION LEVEL > 14 

PLAN TO RE ENLIST » PROBABLY YES 

I FIND MY JOB = FAIRLY INTERESTING 

UTIL OF TAL/TRNG = FAIRLY WELL 

JOB INSIDE US » YES 

SUM TIME . CI V PAY = 99.9910. 

ORGANIZATION/BASE = AEROSPACE MEDICAL DIVISION AFSC BROOKS AFB TEXAS 



DISK 


Per Cent 

TASK TITLE Time Spent 


dt Mfti 1 1 s 1 1 Wii 

Percent 


K 


24 


Prepare Individual Pay Records fur Civilian Employees 


H.OO 


8.00 


K 


3<> 


Procc\s Payroll Cbani>cs tor Civilian 1- mployccs , 


6.67 


14.67 


K 


17 


Make Payroll AiljusliDcnis for Civilian Pay 


6.67 


2I.3J 


K 


15 


Maintain Payroll Control Register for Civilian limployecs 


5.33 


^ 26.66 


K 


25 


Prepare PayrutI Change Slips tor Civilian Employees 


5.33 


32.00 


K 


,U 


l*riK'e^s Time and Attendance Report 


5.33 


37.33 


K 


7 


Compute or Post Allowances. l)-'«iictions. or Differentials fur Civilian Pay 


5.33 


42.66 


K 


\H 


Verify Accuracy of Payments to ( ivilians 


5.33 


48.00 


K 


21 


. Prepare Computer Input for Civilian Pay Actions 


5.33 


53.33 


K 


2K 


PriHress and Post Basic Documents Authori2ing Pay and Changes of Pay for 
Civilian i;n)pluyecs. 


5.33 


58.66 


K 


19 


Pust Service History and Physical Data to Individual Retirement Records 


4.00 


62.66 


K 


6 


Cumpute or Post Allotments for Civilian Pay 


4.00 


66.66 


K 


12 


Maintain Civilian Individual Leave Records 


4.00 


70.66 


K 


13 


Maintain Files of Civilian Pay Documents 


4.00 


74.66 


K 


29 


Process Civilian Cash Awards 


4.00 


78.66 


K 


II 


Issue Civilian Pay Larning Statements 


2.67 


81.33 


K 




Audit Individual Leave Records 


2.67 


83.99 


K 


H 


Cumpute or Post Special Pay for CiviliansSuch as FircPtghtcr Pay 


2.67 


86.66 


K 


10 


Initiate Card Change to Civilian Pay Accounts 


^67 


89.33 


K 


32 


Open or Close Civilian Pay Records 


2.67 


91.99 


K 


22 


Prepare Fmptoyce*s Federal or State Tax Report 


1.33 


93.33 


K 


2 


Balance With Carriers on Each Type of Insurance Option 


1.33 


94.66 


K 


14 


Maintain Insurance Application File 


1.33 


95.99 


K 


20 


Prepare Bond Issuance Schedules fur Civilian Emi^oyces 


L33 


97.32 


K 


36 


Re-estahlisli (*ivilian Pay and Leave Records 


1.33 


98.66 


K 


23 


Prepare i:mploycr*s 1 ederal or State Tax Report 


1.33 


99.99 
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